What's New On These Pages: December 2016

Next month: January 2017

• Saturday 31st December
  After it had run OK today, I turned off the 3.00pm cron job that announces the live and exclusive commentary for the TV show because tonight's show is the last one ever.

• Friday 23rd December
  Worked on tidying some long-standing issues with the Euro Millions auto-grabber. The first problem was that when there were no jackpot winners, this wasn't sensibly detected by the code and it would conservatively back off and not display any of the winners/prizes info. I now look at the UK site's first line of their Prize breakdown table and if I see "Match 5 + 2 Stars 0" there, then I now set the data field to be no Euro Millions winners (Europe-wide).

  The second problem was that the separate script to grab UK multi-raffle winning numbers was looking at the previous draw's info until all the rest of the latest draw's info was in place, at which point it would then switch to the latest draw's raffle winners. Luckily, this would only mean a 2 minute delay, but I now pass the latest draw number to the separate script and it now uses that as the draw number, even if the main data file doesn't have that draw number info in it yet. Makes the raffle numbers appear 2 minutes more quickly now...whoo!

• Thursday 22nd December
  Completed some dummy runs for the main Lotto and Thunderball post-midnight processing successfully, so the cron job for that has been re-enabled.

• Wednesday 21st December
  These draw problems are getting me down in the last week or so! The latest one is that the new code I added on Sunday for post-midnight processing went berserk and zeroed my Euro Millions data file (luckily only losing one draw's worth of info, which I put back in). This gave a very strange draw year of 2004 (time zero for the main Lotto of course) for the latest draw after midnight, but I've corrected that now. It also created a bunch of newsflashes up to draw #1004 for some weird reason (the extra ones have been deleted).

  As a precaution, I now back up the three lottery data files before I start any auto-scraping in case it goes berserk again. I think I've tracked down the issue to the lack of "--date=yesterday" in HP-UX's date command (it's in the Linux/coreutils version). Luckily, I have access to a coreutils version of date on HP-UX which does support that option, so I'm pointing to that now. I also will no longer update a data file if the new version is zero length (yes, an obvious check, but I hadn't put that in until now...duh).

  I've put the fixed code through some dummy runs (which I really should have done when the code was finished!) simulating being run after midnight without the latest draw results and everything seems fine now for Euro Millions post-midnight processing. I've disabled the post-midnight processing for the main Lotto and Thunderball for the moment, but will do dummy runs early on Thursday morning to confirm that the code for those draws is OK too.

• Sunday 18th December
  Yep, another draw problem last night as Camelot didn't appear to release any results at all until well after midnight. I don't normally run cron jobs past 23:58 because the day of the week changes from midnight onwards and my current code wouldn't recognise it as a draw day any more. In these cases, I have to wait for Camelot to finally update and then manually update my data file and regenerate the pages - all horrendously manual and a complete time-waster to be honest.

  Having done yet another post-midnight update, it struck me that it would be quite easy to extend the cron jobs into the next day (e.g. up until 07:58, which is only 2 minutes before online ticket sales re-open and surely the previous night's results must be out by then [e.g. you'd need to know if it's a rollover or not]?).

  What the code needed to do (and I found it quite easy with "date --date yesterday") is that if it's between 00:00 and 07:58, then use yesterday's date/day of the week, which convinces the rest of the code that it's a draw day. I've now made these changes for the main Lotto, Thunderball and Euro Millions auto-scraping. This means any post-midnight official site updates will be replicated within 5 minutes (or 10 minutes for Euro Millions, because that hardly ever releases results beyond midnight).

  The irony of this is that after years of manual post-midnight updates, the imminent demise of the live TV show (last one on 7th January 2017) means that the Saturday night draw will be at a fixed time and it probably never be post-22:00 again like it was last night.

• Saturday 17th December
  Another Euro Millions problem last night as my parsing of the official Spanish site's winning numbers incorrectly picked up the previous draw's numbers :-( This was because I was just looking for "16/12/2016" for the next draw date on the page, but this "next draw date" also appears on the previous draw's page as a future draw date link!

  I fixed it by also looking for the word "Prize" on the same line as the next draw date, which only occurs for the Euro Millions latest results info. In theory, I could check that two consecutive draws have different numbers (the chance of that not happening is extremely miniscule), but what if they actually didn't?

• Wednesday 14th December
  Last night's full Euro Millions results didn't auto-appear on the pages and I didn't check that until early the next morning. It turns out that the official Irish site has had a redesign and my page scraping code wasn't finding anything. It didn't take too long to adjust the code and get it working again though.

Previous month: November 2016

© Richard K. Lloyd & Connect Internet Solutions Limited  2024