Forum

Unstructured Raw Da...
 
Notifications
Clear all

Unstructured Raw Data from Website

4 Posts
2 Users
0 Reactions
76 Views
(@fxxtrader)
Posts: 2
New Member
Topic starter
 

Hi - thanks for the invite, I am glad to be here. 

I am looking to systematically extract data from this website into Excel.  Each Friday this site updates it's data for futures contracts. 

This is the website: https://www.cftc.gov/dea/futures/deacmesf.htm

The information which is relevant to me is the LONG and SHORT values per futures instrument, i.e CANADIAN DOLLAR, SWISS FRANC, BRITISH POUND STERLING, JAPANESE YEN etc (each future contract also has a code #) 

I tried importing through excel Data tab through From Web option. I also tried saving it as txt file and opening in excel through text to columns, but I can't get it formatted as the raw information seems unstructured.  

Any help or suggestions how to extract this specific information:

LONG | SHORT | LONG | SHORT

53,824 40,578  71,858 113,008

Thanks in advance.  
 
Posted : 24/04/2021 10:59 pm
(@mynda)
Posts: 4761
Member Admin
 

Hi Nazaar,

Welcome to our forum! Thanks for sharing the link. Unfortunately, the data on that page is not in a format Power Query in Excel or Power BI can 'see'. 

However, you can use Get Data > From Web. Then at the Navigator select the 'Document' and 'Transform Data'. In the PQ editor window delete the 'Navigation' step. Click on the cog for the Source step and change 'open file as' to 'Text File'. You can then remove the top n rows and go about cleaning the rest of the data.

Hope that points you in the right direction.

Mynda

 
Posted : 25/04/2021 7:49 am
(@fxxtrader)
Posts: 2
New Member
Topic starter
 

Hi - thanks for the quick reply.  I am going to try your suggestions.  I have not used PQ ever, which of of your training vidoes would you recommend to watch to do this specific job?

The data updates each Friday and I need to go through the historical data and pull the stats for about 10 futures contract.  

Thank you. 

 
Posted : 25/04/2021 2:48 pm
(@mynda)
Posts: 4761
Member Admin
 

Hi Nazaar,

If you haven't ever used Power Query it's going to be a bit tricky as you'll need to use loads of different techniques to clean this data including splitting columns, removing rows and more. I cover these techniques in my Power Query course, but I don't have any videos on YouTube that specifically teach these skills, sorry.

Have a go and if you get stuck, consider my course, or come back here.

Mynda

 
Posted : 25/04/2021 9:38 pm
Share: