New Yahoo Price pages - historical downloads - healp with scraping please
Posted: June 26th, 2017, 11:16 am
Hi All,
I used to find it very useful to be able to automatically scrape price history data for given stocks from the yahoo price history downloads.
The syntax used to be :
http:/ /chart.finance.yahoo.com/table.csv?s={EPIC}&a={a}&b={b}&c={c}&d={d}&e={e}&f={f}g=d&ignore=.csv
Where
{EPIC} was the stock code including the .L for uk stocks eg MKS.L and
{a}-{c} described the first date to be extracted ={a}=month, {b} = day {c}= year
{d}-{f} described the last date in the same format
However since the "upgrade" to the yahoo boards the syntax appears to have changed and is now
https:/ /query1.finance.yahoo.com/v7/finance/download/{EPIC}?period1={DATE1}&period2={DATE2}&interval=1d&events=history&crumb={CRUMBVALUE}
Where {EPIC} is as above and {DATE1} and {DATE2} are now unix datestamps.
{CRUMBVALUE} appears to be a 'randomised'(?) string which appears to be session specific, I think it is always 11 charecters.
I've worked out a semi manual way to get the right {CRUMBVALUE} (simply by looking it up before launching the scraper).
Bu my code is still failing, it generates the correct url (which when pasted into the browser produces the download csv) but trying to process this using curl & php is failing.
Before I launch into a lengthy investigation and debug - I wondered if anyone else had encountered and overcome this?
Regards,
Gromley
I used to find it very useful to be able to automatically scrape price history data for given stocks from the yahoo price history downloads.
The syntax used to be :
http:/ /chart.finance.yahoo.com/table.csv?s={EPIC}&a={a}&b={b}&c={c}&d={d}&e={e}&f={f}g=d&ignore=.csv
Where
{EPIC} was the stock code including the .L for uk stocks eg MKS.L and
{a}-{c} described the first date to be extracted ={a}=month, {b} = day {c}= year
{d}-{f} described the last date in the same format
However since the "upgrade" to the yahoo boards the syntax appears to have changed and is now
https:/ /query1.finance.yahoo.com/v7/finance/download/{EPIC}?period1={DATE1}&period2={DATE2}&interval=1d&events=history&crumb={CRUMBVALUE}
Where {EPIC} is as above and {DATE1} and {DATE2} are now unix datestamps.
{CRUMBVALUE} appears to be a 'randomised'(?) string which appears to be session specific, I think it is always 11 charecters.
I've worked out a semi manual way to get the right {CRUMBVALUE} (simply by looking it up before launching the scraper).
Bu my code is still failing, it generates the correct url (which when pasted into the browser produces the download csv) but trying to process this using curl & php is failing.
Before I launch into a lengthy investigation and debug - I wondered if anyone else had encountered and overcome this?
Regards,
Gromley