Profile Picture

Plugin error -Yahoo threw an exception

Posted By CR 6 Months Ago
Message
CR
Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)
Posted Friday May 26 2017
Folks,
when trying to update historical data for NASDAQ-S&P stocks I keep receiving the following error message: "Error: The service plugin RightEdge. Data.Retrieval.YahooDataService threw an exception of type System.Net.WebException".
By chance, coud you please be so kind to give me a bit of a hand?
Thanks indeed!
Cheerio,
CR
Posted Tuesday May 30 2017
Yahoo has 'discontinued' providing data in its current format. We're currently scrambling to find a workaround if they provide any. Will keep you posted.
Posted Tuesday May 30 2017
Just a quick update, the service in its previous state appears to have been completely discontinued with new plans to restore it. This was done without warning or notice.

They appear to have a way to download data that is different, but it looks like the data isn't entirely accurate.
CR
Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)
Posted Wednesday May 31 2017
Bill, thanks! So, can we expect a solution soon, cannot we? Thanks!
Posted Wednesday June 07 2017
Same here just wanted to second this one!
CR
Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)
Posted Saturday June 10 2017
Folks,
any news?
Thanks!
CR
Posted Tuesday June 13 2017
If, like me, you're evaluating RightEdge and want a quick way to get data into it, you can use Google Finance API.  Officially this has been discontinued but it seems to still work, though I think there are throttling limits which means it often only returns the past 1 year of data (to date).

This is how I got it working:
1. Create a copy of the .Net Sample labelled HistoricalDataRetrieval and open it in Visual Studio Community edition.  It will want to upgrade the project.
2. Right-click the project and under Properties, change to target the .Net Framework 4.5.
3. In the HistoricalDataRetrieval.cs file, expand the IService Members section and add the following:

public bool SymbolSourceAvailable
  {
   get { return false; }
  }

  public ISymbolSource GetSymbolSource()
  {
   return null;
  }


4. Now expand the IBarDataRetrieval Members section and replace the RetrieveData method with:

public List<BarData> RetrieveData(Symbol symbol, int frequency, DateTime startDate, DateTime endDate, BarConstructionType barConstruction)
  {
   if (frequency != (int)BarFrequency.Daily)
   {
    return new List<BarData>();
   }

   if (startDate == DateTime.MinValue)
   {
    // I think this is about as far back as Yahoo will go
    startDate = new DateTime(1960, 1, 1);
   }

   // Build the URL to connect to Yahoo using the specified symbol.
   string baseURL = "http://www.google.com/finance/historical?";
   string url = baseURL + "q=" + symbol.Name + "&startdate=" + startDate.ToString("MMM") + "+" + startDate.ToString("dd") + "%2C" + startDate.ToString("yyyy") + "&output=csv";

   // Make the web request.
   WebResponse result = null;
   WebRequest req = WebRequest.Create(url);
   result = req.GetResponse();
   Stream receiveStream = result.GetResponseStream();
   Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
   // Read the data from the stream.
   StreamReader sr = new StreamReader(receiveStream, encode);

   string pageString = sr.ReadToEnd();

   sr.Close();

   List<BarData> bars = new List<BarData>();

   // Parse the request into bars.
   PutRawDataInBarCollection(pageString, bars);

   return bars;

  }


5. Build the project.

6. Copy the DLL to the /plugins folder in the Right Edge program folder

7. In Right Edge, use Tools > Configure Services to add this service as a New service (may require Right Edge restart to pick up the new DLL I think)

8. The new service can be assigned to a folder in the watchlist and should now load data.  I have it working here.

Hope this helps.  I'm just a newbie to Right Edge so may have missed better ways of doing things. This was just a quick 'hack' to try to get up-and-running while I evaluate the platform.


Posted Tuesday June 13 2017
Impressive piece of coding from someone just evaluating RightEdge ! The code worked for me too thanks vm for posting it. I knew from other blogs - like quantmod - that Google was an option but I would have thought it would be harder to get working. Maybe I am just lazy .. 

I thought I would share a couple points. I modified YahooDataRetreival directly - and switched it to google (note above point about lazyness) - it seemed easier to me than creating a new service. Though your code kind of looks like a copy of Yahoo too.

Google does not have Adjusted prices like yahoo does - ie they are not adjusted of divs and splits etc - so some people see google as inferior. For what I do it is ok for now. Better than nothing (or IB slow historical feed which was killing me )

I had to make sure to untick 'use adjusted prices' in the service options for Yahoo when I ran the new pluggin. Otherwise the code would be checking a value that does not exist.

Also as the google file has one less column there is a place in the code that checks for columns = 7. I changed to 6 and all worked smooth from there. 

Thanks again
J

Posted Wednesday June 14 2017
Ah, yes, I completely forgot to mention the 6 columns rather than 7.  I took a slightly different approach and just hacked the code to use AdjustedClose = Close.
For anyone not wanting to do the coding, here's the method you need to replace with the column numbers corrected:


private void PutRawDataInBarCollection(string webPage, List<BarData> bars)
  {
   string[] lines = webPage.Split('\n');

   foreach (string line in lines)
   {
    string[] items = line.Split(',');

    if (items.Length >= 6)
    {
      double temp;

      IFormatProvider culture = new CultureInfo("en-US");

      // need to set culture info for non-US folks since we're getting
      // data from a US source
      if (double.TryParse(items[1], NumberStyles.AllowDecimalPoint, culture, out temp))
      {
       BarData bar = new BarData();
       bar.BarStartTime = DateTime.Parse(items[0], culture);

       // We need to calculate a factor so we can
       // to normalize the open/high/low/close.
       // This is necesary because of splits.
       double adjustedClose = double.Parse(items[4], NumberStyles.Number, culture);
       double realClose = double.Parse(items[4], NumberStyles.Number, culture);
       double adjustmentFactor = realClose / adjustedClose;

       bar.Open = double.Parse(items[1], NumberStyles.Number, culture) / adjustmentFactor;
       bar.High = double.Parse(items[2], NumberStyles.Number, culture) / adjustmentFactor;
       bar.Low = double.Parse(items[3], NumberStyles.Number, culture) / adjustmentFactor;
       bar.Close = double.Parse(items[4], NumberStyles.Number, culture) / adjustmentFactor;
       bar.Volume = ulong.Parse(items[5], NumberStyles.Number, culture);

       bars.Add(bar);
      }
      else
      {
       // Report an error!
      }
    }
   }
CR
Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)Supreme Being (494 reputation)
Posted Friday July 28 2017
Hi there!
Any news?
I have already bought the perpetual license and now the stuff is not working....
I would highly appreciate your kind assistance.
Thanks -have a great weekend,
CR


Similar Topics


Reading This Topic


2005-2017 © RightEdge Systems