Steam Group Data Mining
During my latest attempt to avoid Uni work, I decided that I would try and code up a couple of PHP Functions that would basically output everyone in a given steam group’s top class in TF2, so for me it would solider with the amount of time in Hrs, mins and secs.
There are a few little things, the fact it can take ages to get through all of the members of a group, Im not sure if its the way I have gone through each user, or just how slow the steam servers can be. Also, if a user has their profile set to private their data will not show up.
Basically the first function just converts the seconds it pulls from the Steam Community XML file into hrs mins and secs. I did get this function from a nice website (that I cannot remember 
)
//Converts Seconds to HHH:MM:SS function sec2hms ($sec, $padHours = false) { // start with a blank string $hms = ""; // do the hours first: there are 3600 seconds in an hour, so if we divide // the total number of seconds by 3600 and throw away the remainder, we're // left with the number of hours in those seconds $hours = intval(intval($sec) / 3600); // add hours to $hms (with a leading 0 if asked for) $hms .= ($padHours) ? str_pad($hours, 2, "0", STR_PAD_LEFT). ":" : $hours. " Hrs : "; // dividing the total seconds by 60 will give us the number of minutes // in total, but we're interested in *minutes past the hour* and to get // this, we have to divide by 60 again and then use the remainder $minutes = intval(($sec / 60) % 60); // add minutes to $hms (with a leading 0 if needed) $hms .= str_pad($minutes, 2, "0", STR_PAD_LEFT). " Mins : "; // seconds past the minute are found by dividing the total number of seconds // by 60 and using the remainder $seconds = intval($sec % 60); // add seconds to $hms (with a leading 0 if needed) $hms .= str_pad($seconds, 2, "0", STR_PAD_LEFT) . " Secs "; // done! return $hms; }
This next function basically mines through a steam group, to get each of the members then pull the steam id and top tf2 class from each of them and save it into an array.
function topClass($steamGroup) { unset($playerStats); unset($xmldoc); unset($xmldoc2); $xmldoc2 = new DOMDocument(); $xmldoc2->load($steamGroup); //Gets number of users in group $members = $xmldoc2->getElementsByTagName("memberCount"); $memberCount = $members->item(0)->nodeValue; $count = 0; $SteamIDs = $xmldoc2->getElementsByTagName("steamID64"); //For each member get top TF2 class while($count < $memberCount) { $currSteamID = $SteamIDs->item($count)->nodeValue; //Steam Profile $steamProfile = "http://steamcommunity.com/profiles/".$currSteamID."/?xml=1"; //my steam data $xmldoc = new DOMDocument(); $xmldoc->load($steamProfile); $name = $xmldoc->getElementsByTagName("steamID"); //Gets Steam Name $steamName = $name->item(0)->nodeValue; //Steam Stats for TF2 $steamTF2Stats = "http://steamcommunity.com/profiles/".$currSteamID."/stats/tf2/?xml=1"; $xmldoc2 = new DOMDocument(); $xmldoc2->load($steamTF2Stats); $className = $xmldoc2->getElementsByTagName("className"); $classTime = $xmldoc2->getElementsByTagName("playtimeSeconds"); $topClassName = $className->item(0)->nodeValue; $topClassPlaytime = sec2hms($classTime->item(0)->nodeValue); //Inserts data into Array $playerStats[$count][0] = $steamName; $playerStats[$count][1] = $topClassName; $playerStats[$count][2] = $topClassPlaytime; $count++; } //returns an array of Steam Names, class and play time. return $playerStats; }
And last but not least, a function to print it all out, its not very pretty, feel free to add CSS or have it print out some other way, atm it just prints out into a big table with headings, Name, Class, and Time. Simple for the Clan leaders who just need the basics like myself.
function printClassTable($steamGroup) { print "<table border=\"1\">"; $playerStats = topClass($steamGroup); $numElements = sizeof($playerStats); $count = 0; while($count < $numElements) { print "<tr>"; print "<td>".$playerStats[$count][0]."</td>"; if($playerStats[$count][1] == NULL) { print "<td></td><td>Private Profile</td>"; } else { print "<td>".$playerStats[$count][1]."</td>"; print "<td>".$playerStats[$count][2]."</td>"; } print "</tr>"; $count++; } print "</table>"; unset($playerStats); }
And there you have it a rather simple way of getting an entire steam group worth of data. I have commented the functions pretty well (i think…) so that anyone with some knowledge of PHP should be able to work out what is going on and make changes to suit them.
Feel free to ask questions by leaving comments!
Thought I would add an example i did from the TeamDamage steam Group (Hopefully no one minds i used that group! 
)
TeamDamage Members Top TF2 Classes (Keep in mind it may take a while to load!)
P.S: Thank god for WP plugins for posting source code for stuff! 
A day at TeamDamage…
It was a rather long day, especially when you dont get in until 2am the morning before, then gotta be at Unanderra at 9am. It kinda takes it out of you. It was fun nonetheless with heaps of fun stuff going on. But I think the most fun part of the night was going head to head with a couple of veterans of Quake 1. We had people in fits of laughter by the shit we were saying to each other while playing. It was just a whole lot of fun. The other games were good as well I suppose, but we are constantly having the issue where team balance is way off. With games like TF2, it is really hard, because us D! players who play on a regular basis and are used to playing in a competitive environment, though we did split the D! players, it still turned into a rather one sided game, since you have people there from all walks of life, some who may only play the game while at TD, other likes myself play it a hell of a lot. I am thinking that we may be able to fix this by using one of the different game modes, like king of the hill or arena, since they are completely different game types that the more regular players may not have played outside of a pub environment. Which i think, if we were to play TF2 in that way, it would help fix some of the issues that we have with balance, though there would still be some advantage, it may not be as big as it was today. With games that are popular, there is always a skill difference, and if one team gets smashed and doesnt put up much competition it just ruins it, since its not as good a game. Sure, there are a number of other games to play, but getting people who have them, with the correct version and basically ready to go straight away is hard. Steam offers this really easily, since everyone’s TF2 needs to be up to date to even run, we dont run into those issues. I remember back in the old days of TD, we used to have huge issues with versioning, half the people would have the correct version, the other half would have another version and people would spend the next 4 hrs working to try and get the game working. Games have come a long way since that issue, most having auto updaters that keep the game up to date, but some of the younger LANers, dont see this as an issue, where people who have been around for about 5 or 6 years, know that the best way is to have a game that will just work, normally it is by playing it on a base install or making sure everyone follows the exact procedure to install the game and patches, I remember having to do it on more than one occasion.
If you think about it, LANs have become a bit of a endangered species, since the internet has gotten fast enough to play games over the internet with little lag, LANs have lost of lot of their usefulness. The question we are often faced with is how to we attract new people into a LAN, when most of the gaming you can do there you can do from the comfort of your own home. The main thing is the social aspect of a LAN, playing gaming with people over the internet and using teamspeak is one thing, but being able to walk over and talk to the person after a game and discuss it in person has its appeals. Question is how to we get people to recognise that and come to the LANs.
Over the many years I have attended LANs on and off, some of my best memories are from the old TD at the PCYC where we would start on a Saturday afternoon, and go until Sunday night, LAN all night, some random stuff would happen during those night, especially involving the foam pit in the gym and the pool noodle attacks on moddy. There are some major differences from the old TD to the new, sometimes I think we have lost something that made TD great, the random fun we would have in the middle of the night inside the PCYC. I am starting to think, that fun social part of TD needs to somehow be integrated into the new TD inorder to bring more people back. You need to set the LAN apart from things you can just do at home. Im not saying the new TD isnt fun, but its that randomness that made TD great, and in the early days, kept me coming back, time and time again. TD now, is a lot more structured and we play a lot of games that are well formatted (apart from the balancing issues), but I cant help but think, that we are moving away from that random fun side. I know I am just criticising and have no idea how we could get that random fun element back into TD, but I think we need to do something that will attract people who will come time and time again.
It just goes on…
Posted by wrighty in Computers, Personal, University on August 13, 2010
You know that feeling you get when you look down at a test after you trick yourself into believing you know the material and then look at the paper and think, oh shit I don’t know that. It happens to me a fair bit unfortunately, sometimes because I deserve it for not really studying or caring about the test, and sometimes I genuinely study and try hard and still fall on my face. Today’s test was one of those times where I wasn’t caring and I am more than likely going to fail it because of that lack of caring. As of late, university has been going from one pathetic class to another. Poor tutors, boring lectures and complete fuck ups. Looking at the past 3.5 years of my degree, I have done some 28 classes, with the majority having something that could be complained about. I know what your thinking, I am just whinger, which may be true for some of the more tolerable classes, but some of the classes I have done over my time at university, are genuinely crap and are in need of some major help. I did a java class in 2nd year, and I went to a handful of lectures and as a result of not being able to understand the lecturer and my belief that I could better learn the java programming language myself through online tutorials which and doing that I was able to complete the class with a HD. The tutor for that class spent the time doing his own assignments, and learning Java. Its amazing, a University which claims to be the best, has such socking teaching standards that some students have trouble trying to understand what is the lecturer is saying rather than learning the concepts of the course you are paying around a thousand dollars for.
Ok, I suppose thats enough bitching for the moment. On the good side I brought the new PSU and HDD and installed them. So good so far, nothing has blown up, but i am still a little iffy on the hard drive since I am worried the new 1.5TB drive is going to end up having the same fate as the 3 TB ones before it. Only time will tell i suppose….
The backup begins…
I start this blog entry off with some good news (about time!), I managed to get the majority of stuff off the TB hard drive that was making some suss grinding noises. I got the important stuff, which is what matters in the end. I am kinda lucky the ATO finally came through, it gives me a chance to make some upgrades to my PC, nothing overly drastic, just a new PSU something that isnt some no name PSU that cant handle anything. A lot of people do not realize how dodgy the generic PSUs are. I know I have been lazy, since when I got my new computer a little over a year ago, I didnt get a decent power supply, and now I am paying the price in HDDs (assuming the PSU is the problem of course), so tomorrow hopefully Ill buy a new one, and maybe a HDD as well, this time not one of those green ones, I want one that eats trees and pollutes, at least those dont break. Going green is all good and well, but if they tend to break down, Id rather go non-green, and keep the data longer, otherwise the hard drives keep breaking, they go to land fill and the problem shifts, from a power issue to a land fill issue.
On a less useful topic, I have been doing a bit of thinking lately, it helps me avoid assignments, mostly thinking about the road not taken, like what would of happened if I had elected to go to tafe over uni, where would I be now? that is just one of the what if’s that have crossed my mind over the past couple of days. Some people say what’s done is done, which is true, we have yet to create technology that will allow us to move backwards in time, but it cant stop you from thinking of possible futures that may of been open to you if you had made another decision. Just think, I have made any number of decisions over my life, these decisions have lead me to this point in my life. A point that has made me very much a loner and basically have made me a person who always takes the safe road, the road that will result in a predictable outcome. Though I have come to a point where I have lost the ability to do much else, but do the safe option. Taking the safe option I have sacrificed a lot of opportunities that may have made me a better person in the long run. It has lead me to being a loner that likes being alone, or at least thinks that because it is easier than the alternative. The sad thing is, this way of thinking and acting have lead me to be nice and take a lot of shit, regardless of weather it has anything to do with me or not, I have been used on many occasions and I let it happen. I know deep down im being used, but having someone like me has been more important than the fact I was being used. But its my own fault and im the only one to blame.
I dont know why, but all these posts always end up with some emotional stuff, sometimes i wish I could remove emotion, it would make life a whole let less complicated. But, i suppose this is how I am, deal with it eh?
University, Interviews and a realization…
Posted by wrighty in Personal, University on August 6, 2010
Well it has been one of those weeks, missed a phone call from Air Services, for them to ring me back at 9am while I was half asleep for a Phone interview, got an email from DHS and 2nd week of uni. Dont you just hate the feeling you get when you know you didn’t do well in an interview. I was tongue tied for the entire time and it took until I was on the phone to realize that I have no idea what Air Services actually did, sure I had a ball park idea, but nothing concrete. I am pretty sure I did almost everything that could go wrong in an interview. Sometimes I think I should answer the “Why do you want to work for us” question with, because you have a graduate program that pays me money and will probably put my degree to use, though I dont think that it will get me a job, but i think i would be the most honest person they talk to for the entire day. For those interviews you need to come up with a good original way of saying why you want to work for them, regardless of how you really feel.
Also this week I finally got something from DHS, though it was an email explaining that they had been delayed, its nice to have something concrete that I have an offer from them. Though it has made it a little more real, the fact that at the beginning of next year, I will be moving out of home for the first time and moving to a brand new city. If someone had told me 4 years ago that I would be moving out of Wollongong after graduating University, I wouldn’t of believed them, and now it is slowly becoming very real and im coming to the realization that more than likely I would be moving out on my terms, but more on take what I can get so I can work and get money terms. I always hoped that I would of gotten a job in Sydney, travel on the train up and down for a while, save up then move closer to Sydney. But that is out the window, and at the moment, once I am in Canberra ill be on my own for the first time ever, and I dont mind saying that it does scare me a little. I have never been the most outgoing person, as most people would of realized by now, moving out of home so quickly and without a decent support network is a little stressful for me. I suppose most of you would think its great to be getting out of home and moving to a new city, and part of me does think that way, but I also am thinking of everything else, the money related stuff, as well as short term getting around. As it is the barina wont be able to make it there (let alone back) so ill be stuck without a car until I can afford one, which I think its going to be a decent amount of time, probably after the shock of moving out and getting into a new routine has subsided. I do have a small amount of hope that somehow, magically no doubt, get a car that I can drive around for my first few years down there until I can afford one, but im not holding my breath. Without a car, the short term getting around in Canberra is going to be a major issue, especially in a city I have no idea about the public transport system.
Now, Uni, the first week, pretty cruisey as normal only lectures no tutorials, second week though, normally has some interesting bits and pieces, especially in project, since they fucked up the autumn session part of the subject, the new coordinator are in some major damage control trying to fix ALL the subjects issues, which is almost impossible without restructuring the entire subject. This subject was a complete waste of $1200 (more than that actually), it basically was a report… for a case study we had to buy ourselves from the Harvard business school, and that the marking criteria was not released until week 7. So the first 6 weeks of this project were based around speculation of what he may of wanted in the final report and presentation. It took our tutor writing up a criteria and handing it to the lecturer in order to a get decent idea of what we needed, so a 12 week report was basically cut down to 5 weeks or less including the time you needed to put it all together. Then they wonder why the reports were seemingly incomplete, maybe its the fact that we had no idea what he wanted until 7 weeks into the assignment. Then, when you think the subject couldnt get any worse… the exam… WORST RUN EXAM I HAVE EVER SEEN, first, the exam was given to another class (exact same exam) a whole hour or two before our class was set to take the exam, next… the lecturer turned up to the exam (which was in the lecture hall) short by about 20 exams, so 20 people got to look at the first 2 pages of the exam question, before being told they would have to take the exam at a later date since putting the exam on the project was no going to work. If these two major issues weren’t bad enough, the fact that there was a lot of cheating, and no clear outline of what would be in the exam other than 10 weeks worth of lectures… seriously even the hardest lecturers would give clues about what would be in the exam, though when he did give us a little heads up, he added to the end that “I believes that what ill put in the exam, when I actually write it”, so a week before the exam, there was no exam even written. Now if we fast forward back to the tutorial this week, they had us do a survey on what we though of the subject, a few of my more opinionated friends took the extreme point of asking them to fire the lecturer, which is never going to happen, though I thought it was funny that they wrote it on there. Sometimes I wonder where the quality control is for the University, I have been through some of the worst run subjects, if I had walked into a shop and got a dodgy product, I would take it back and ask for a refund, now where is that kind of ability to ask questions and go is this really the standard that we should accept, or should we be able to get a discount for poorly run subjects??
Well then, yet another rant from me… This one was kinda long, and I am writing it at about 12.30am, when I have class in the morning at 8.30am… Boy, am I going to have fun getting out of bed in a few hours.
And…. another hard drive bytes the dust!
HA HA… ITS A PUN!
back to the Blog post….
As you no doubt have figured from my title, I have lost yet another HDD, a TB hard drive that stores a lot of stuff. If you haven’t keep up to date with my different computer issues with hard drives lately, In the past year and a half I have lost 3 separate TB hard drives. The first 2 being seagates and the most recent being a Western Digital. The Western Digital drive has lasted the longest with a good 7 months under its belt. It is just annoying that that I am going to lose around 500GB of stuff. A complete pain in the ass that annoys me to no end!
The Question is, what do I do to fix this issue of Hard drives fucking up in my computer, new PSU and Hard drive.. keep better back ups?
What makes it worse is I happen to store all the digital photos on that drive. On the upside I do keep a back up on another computer on the network, and I happened to make sure that backup was up to date on the weekend (thank fuck). The rest of it I can get back through other means, and its not that important. But I am beginning to think i need to be a little bit better on the whole backup front. You think its funny, I tell everyone that they should backup and make copies and stuff and I hardly ever follow that advice.
I suppose now i have something to spend some tax money on, even though I have received that money yet due to more and more issues with their IT systems that have costs god knows how much money over the last few years, who knows I may be working for them one day! On top of that I didnt really want to spend the money since I will be moving to Canberra at the end of the year and will need that money to help me get settled. Thinking of the whole Job in Canberra thing, I still haven’t received the “offer package” yet. I kinda expected it to be here by now, but it still hasn’t made it here. Its kinda strange, but I drafted an email last week to send to them but I thought it had only been 1 working week since they had called and said they would send me out one. So i decided that I would give them another few days to send me out. So ill send the email in a few more days hopefully.
The Ubuntu Download Manager Script
Now, regardless what the title says, I have done my best to create a download manager for Ubuntu Server, that is based around a perl script and php web pages that allow the addition and removal of links that you wish downloaded. It is no where near perfect, and im sure there are people that could do a way better job than I, but it works for me, and I am more than happy to provide the perl script and PHP pages to anyone who would like to see the script.
The History
I created this script, when i converted my download box to ubuntu server. I wanted to have a more stable system, also wanted to run it on a low end system without the resources being taken up by GUIs and other fancy stuff. Disappointed in the lack of download managers available that would give me command line or web access to it. I took it upon myself to create my own. Using knowledge I had learnt from University and my own little projects in PHP and perl, I began creating a script that would download files using originally using wget (since has changed), I created the script to download files from Rapidshare using a premium account, but since I extended it to download any type of file as long as you have a direct link. With me being at University, it has taken me a long time to be able to complete the script. But with it running and a number of support scripts such as extraction, init scripts and sbin scripts that all help control and give additional abilities to the script such a pause and resume.
The Mechanics
Basically, in simple english, the script runs in the background, it reads a database table for links, and downloads them one at a time. The script will check the database every 120 seconds if there are no links.
Features
- Can download Rapidshare Files (Assuming you have an active premium account)
- The ability to define a save directory or have the script do it automatically
- Pause and Resume downloads
There are a number of additional scripts I have written to support the perl script, including init scripts, sbin script and extraction scripts. These all support the main script.
Future Updates
There are a few things I wish to add in the future, including more control over what can be done from the webUI, I wish to add the ability to extract files and control the script (Start, Stop, Pause and Resume).
If you have any questions Ill be happy to respond to them, leave a comment or email me (wrighty@thewrighty.com)
The University… never ending ways the piss me off
Posted by wrighty in University on July 25, 2010
Well Uni starting next week, I have been confirming my timetable just in case they changed something, and looking over subject outlines to see what I will be doing this session. When I found to my surprise, that they had infact changed the timetable for one of my classes, pushing the class back an hour. This wasn’t too bad considering it only means I need push everything back on that day. But it opened up room for a subject that I had considered but didnt enrol in since the times conlicted. Now, I can take this subject (that has no final exam) but the chances on getting a tutorial that will no conflict with work or other classes is going to be almost impossible since tutorials would likely be full. God, the last 2 sessions have been time table fuck up, we had 3 room changes last session due to timetable issues, and this session an entire class had to have all its tutorials, labs and lecture pushed back an hour. Jesus, you’d think they have been making timetables for so many years, they would have a system that would work… I suppose there are circumstances that fuck it up.
The next issue with Uni this session, is that I need to go back and do a 2nd year subject. For a 4th year student, this is a little weird. Due to the removal of my degree from the program, they have been changing the subjects to accomodate the new B IT degree. Which is all well and good, until I begin redoing subject content that I have already completed in a combination of 3 other subjects that went way into depth into these concepts and in most cases were interesting subjects. Why must I spend 12 weeks, going over stuff that I have learnt before and wasting time in my final year. I couldnt stop laughing when i looked at the lecture schedule and saw that in the 2nd week, the required reading was basic unix commands. Ever since I came into uni, I have been using Unix based systems (normally ubuntu), by second year, I had a firm grounding of the basic commands. If 2nd years in the new IT degree dont have a good idea on how to do those commands, and combine them to get information they require, I do worry about the future of IT. Kinda reminds me of 322, where we had a bunch of people in the IT degree, who didnt know how to code, or could even be bothered learning perl or python, for the assignments. We had to learn perl as part of server technology class, why couldnt they learn it…
Ah well, theres my Sunday morning rant!
The new Dedicated Blog Sub domain!
Well, tonight I got bored and decided that I wanted to do more of this blogging thing. Which lead me to create a sub domain (i do love doing that) and installing a CMS that is dedicated for blogging. As for the old blog… well It is just going to be disabled and forgotten for the rest of eternity or until drew deletes his web hosting what ever comes first. But until that changes, this blog will be updated occasionally, when I have time, over the next few weeks there will be posts on interviews, uni, work and even on occasion, personal items that I believe the world would be not worse off for reading. I may even post things about little projects I have on the go, like the downloading script/web ui that I have been working on.
I will try not too bore the few readers that may come here (if any do) but sometimes, you just need to say it, and let people deal with it, but blasting comments online.