New Stream Uptime Monitoring System

This is more of a technically related article than anything else, but it interests everyone because it has to do with an important element of the stream: uptime.

Uptime is what fraction of a given period of time something is active or running normally, often times expressed as a percentage. In the case of Gensokyo Radio, uptime has been lower last month than it was the month before due to some intermittent software issues mostly out of our control.

There have been a couple of ways that we’ve attempted to cope with the server’s odd behavior up to now. First, we set up a monitoring system on stream data, and that told us if information (such as song title, artist, album, listeners, etc) was showing up where it should. When that data didn’t appear, it would send us (myself and Auritribe) a message letting us know that the server was down and intervention was required. Essentially, it was a basic notification system that couldn’t do anymore than that.

A couple days ago, I installed a more powerful tool which not only notifies us of the server’s status, but automatically intervenes and restarts the server. Figuring out how to do this has been more complicated than it should have been. A sysadmin or another otherwise technically inclined individual might say, run a cronjob. That was one of the first things we tried, but to make a long and boring story short, it didn’t work.

So, why is this a good thing? It’s quite simple: uptime will excel to near its maximum potential with the improved system. Previously, downtime would last anywhere from a few minutes to several hours, depending on how inopportune the timing was (obviously, if an admin is sleeping, they won’t get the email until after they wake up). With the new system, we’ve seen the stream recover, consistently, less than 20 seconds after the server stops responding.

I’ll tell you firsthand, this is a large weight off of my shoulders, and (hopefully) the end of a very annoying issue we’ve been dealing with for the past month.


User Comments (migrated from legacy news system)

rtaglem | Posted on May 10th, 2012, 10:11:08 PM
Thanks for all this man… really… lo aprecio bastante ya que siempre e sido un gran fan de touhou :slight_smile: sigue así! mis grandes y sinceros apoyos hacia ti

LunarSpotlight | Posted on May 10th, 2012, 10:20:04 PM
You’re welcome.

rtaglem | Posted on May 12th, 2012, 05:25:19 PM

Nemo Oceansoul | Posted on May 17th, 2012, 02:32:12 PM
uhh are you sure, its been down since yesterday where im at. its saying its currently playing “tenpest” but it hasn’t changed that since yesterday. is there some wierd unknown glitch or is it just the school computers here? (not blocked Yays :D)

(Guest) htfkid2000 | Posted on May 17th, 2012, 04:10:13 PM
@nemo it’s not a glitch, it’s the same on 2 different internet sources… the server is down…

auritribe | Posted on May 17th, 2012, 05:21:49 PM
“GensokyoRadio Server is inaccessible on our end, we’ve notified the admin and will get it back up when we can” Unfortunately, this downtime is beyond our control. We’re working to get it back up and running, but the whole server itself appears to be down. At the very least, it’s completely inaccessible at the moment.

LunarSpotlight | Posted on May 17th, 2012, 07:24:17 PM
That’s from the Twitter feed. Even if you don’t use Twitter, it’s on the sidebar on the front page, and we use it to announce status messages as well as other smaller bits of news that don’t merit an entire article on the site. I’ve spoken to the people who have higher access to the server again today, and they should be getting to it soon. As for the live info saying the same song for the past 24 hours, that’s due to the system not pulling any new info from the server (which is down). Normally I would manually change it, but I’ve been hopeful that the server would be up within a reasonable amount of time. I’m sorry the server admins haven’t been responsive, but they’ll get access back up when they can, and from there, I’ll start things back up.

Nemo Oceansoul | Posted on May 17th, 2012, 07:33:06 PM
thx. i just recently found this site actually (which makes me happy) but i realize computers can sometimes make even cirno look smart… which is sad… well lets hope it come back soon <3 on another note that is totally out of topic: quick question: when do you think flan-flan will become a playable character in the official games? or okuu… (loves overpower)

LunarSpotlight | Posted on May 17th, 2012, 08:17:18 PM
Well, Utsuho is playable in Hisoutensoku. As for Flan, no idea :3

Nemo Oceansoul | Posted on May 22nd, 2012, 04:45:46 PM
YAYS ITS BACK UP :slight_smile: and with yuyuko’s theme playing too. <3 also: to lunar: yes well when i say “official” i mean the shooters not fighters… i can just imagine how hax okuu would be for the shooters tho. (star sized bullets…)