Power Inquiry was developed particularly to assist users import data from several flat-file databases (i.e. CSV files) or other Excel files, eliminate file headers and footers as required, eliminate columns, rename columns, and add or delete rows.
Practically every step you require can be completed with Power Inquiry and without VBA code.
With Power Query, you will be able to query specific rows from your source file. You could define only the rows that match your 5 or six well identifiers. In this way, you will just return to Excel the wells that you seek, and it won’t matter if the row numbers alter from file to file.
If you desire to pull all wells into Excel, and then massage the data within Excel, you would still use Power Query, but you would not limit the search requirements.
Rather, you would query all rows of data into a single tab of Excel http://www.iditect.com/tutorial/excel-row-column-range/, and then create a second tab for the summary information. On the summary tab, add your well identifiers and then utilize VLOOKUPs, SUMIFS, and other comparable lookup and aggregation functions to pull the data into the summary tab.
Keep in mind that it may be more time reliable to just brute-force it and do the work by hand if this is just a one-time project.
Unless the file formats are well understood beforehand, automating it would be a continuous process of running the data, inspecting the outcomes of the automation, and tweaking the automation.
10 thousand files is a lot, but if it takes simply one minute per file to obtain the relevant data, you’re talking about two man-weeks of effort. Not brave in the context of a dissertation effort, and you can probably get some low-cost assistance if you’ve got a budget plan, through something like Amazon Mechanical Turk or eLance.
It would depend upon the version of Microsoft Office. Excel 2010 enables PowerPivot to be set up without much efforts, Excel 2013 does so for service volume licence (as I far as I know). This tool ought to well handle your query within Excel
For any variation listed below 2010, you can either use VBA if you wish to stick to Excel or you might desire to transform your information files (perhaps using VBA to rearrange you data easily or VBS for this part) into csv files and utilizing the load data mysql function, you may wish to automate this with Python if you have everything in a single folder and if it would take long to by hand import each files.
Editing big quantity of information will slow down your computer if you wish to erase or include columns to it. I use the match/index mix. All you need to know the header name, say in 5 headers you are interested. Compose them down on a different sheet (I prefer to work in the exact same workbook). Now use the match/index combination. It’s method quick and you don’t have to bother picking and deleting any column.
Java can access the Excel utilizing POI, but its handling data type and data computing (such as conditional filtering, organizing and signing up with) is made complex. Here I suggest esProc to you. It needs simply a line of code to access the Excel. You can use it as java runs and it is simple for Java to call it. For computing the Excel information and composing the lead to the Excel, esProc’s code is rather simple.
You can embed esProc in Java to send somebody else, or call it utilizing straight the operating system’s command line.