Perhaps a small introduction is in place here: I’m a human movement scientist by education, so my programming experience is mostly limited to data processing with Matlab. However, since Matlab would set me back at least 2000$ I decided to give Python a shot!
Python’s syntax is very similar to Matlab, making the transitioning a lot easier. One of Matlab’s strong points are it’s vast library of functions for scientific, but luckily that’s another area where Python is a good fit: NumPy and SciPy probably provide every tool I need for most data processing.
However, not all things were well in the start. Because if you don’t know anything about Python, what’s the first thing you do? Right, you install the newest version of Python: 3.1 Shouldn’t be a problem eh? Well until you start trying out coding examples that involve NumPy and get cryptic error messages that you don’t understand.
What’s the problem? Turns out that updating all the Python 2.6 based NumPy/SciPy code to 3.1 is one hell of a job, so that wasn’t available yet. Bummer… I decided to look for a solution that made sure everything was compatible and since I’m on Windows (thus lacking sudo installs), one that makes sure it includes everything ‘out of the box’.
I ended up picking Python(X,Y), which by their own description is a:
Python(x,y) is a free scientific and engineering development software for numerical computations, data analysis and data visualization based on Python programming language, Qt graphical user interfaces, Eclipse integrated development environment and Spyder interactive scientific development environment.
Well that surely sounds like everything I need! It even comes with a load of additional plugins, which might come in handy some day.
So now I had the right tools, I could start cracking right? Well, guess again? Even though Python’s syntax is fairly similar to Matlab, that doesn’t mean you can keep typing Matlab-like code and expect it to work in Python.
Python for instance, doesn’t need counters for loops since lists and strings themselves are iterable. It’s all these little ‘bad’ habits that you have to stop doing if you want to write Pythonic code!
After even extracting the 3D matrix from the software’s export proved to be quite a trial. How do I loop over a file?
for line in file:
print line
Ok that does the trick, but I don’t know whether the line will be a header, the Frame ** number or the real matrix. Ok, I’ll first look for Frame and then start reading adding lines to the matrix from the next line. But how do I find out what line I’m at? Argh!
Well after a dozen or so of these ‘problems’, which made me read tutorial after tutorial and just reading a 16 Mb ASCII file took over 40 seconds, I decided I couldn’t hack my way through this. Time for some proper introduction to Python:
I got my hands on a couple of Python books and decided to start with Learning Python by Mark Lutz. Quite the eye-opener on some topics, though I got most of the basics from several tutorials online. I must confess that I didn’t finish reading the whole book as I didn’t expect myself having to handle Exceptions in Classes any time soon. But I instantly noticed I got a lot more proficient at what I was doing and getting results faster.
However, after about a month of experimenting, I’m still getting caught offhand by obscure documentation and overall lack of programming experience. I constantly feel like I know there’s a better (probably even built-in) solution to problems and like I’m reinventing the wheel. Luckily, this is where Stack Overflow comes in handy! Which is site where you can ask programmer questions for any language and for any level of programming experience.
Now as some might have guessed the name of my blog, I’m a moderator on Super User (one of the Trilogy sites), where I have the honorable task to help out others with their computer problems. To get elected as a moderator, you need to positively contribute to the site and are expected to be an experienced user. The downside of being an experienced user is that it gets increasingly more difficult to ask good questions, since you’re burdened by your own experience and high standards.
However, after running into some problems with my peak detection (looking to distinguish between toes on each paw), I finally made myself post a question on Stack Overflow. Well to say I got great responses would be an understatement, with some of the results looking very promising. The enthusiasm of all the other users is also the reason I’m now writing this blog
I plan to ask more questions in the future, as there’s a lot of room for improvement and this question finally got me over my initial fear of asking newbie questions, so I’m curious to see where we’ll go from here!