Goodbye lazy 3rd world living; Hello lazy 1st world living

Here is another rather personal update.

So, I spent the last year in the poorest parts of South America. Just me, my math books, and 3rd world living. The highland towns of Bolivia are the ones I remember best. No hot water, no heating, sleeping with all my clothes on at the same time. The narrow broken mountain roads, the speed crazy coca chewing bus drivers, all those rickety 16 hour bus journeys. Eating every day at the town markets, sitting next to short and stocky native indians. Two dollars a night for a room, two for food, two for beer and two for internet. A simple but free life. A year of studying.

And then my money ran out. So I’m back in the UK. I found a job in Basingstoke. Its a C++ job. Its traditional software development (as opposed to 24/7 web service plumbing) just like I wanted . The hours are 9to5 and the pace is quite relaxed. The pay is good. There is some maths in there as well. This looks like the best job I have had so far. In my previous jobs I always ended up working 10 hour days. I dont want that again. There is more to life than work. I think now I will have time to study and enjoy my free time a little. I also went to a couple of interviews for investment banking jobs, but backed away when I realized it was exactly the type of job I didn’t want. I have had enough of maintaining large fragile legacy 24/7 systems. I can do without those 3AM support calls, thank you. I’m happy I did not end up in Games either. Long hours, low pay and tight deadlines. No thanks.

I’m reading a book now called “The C++ FAQ 2nd edition” by  Marshall Cline. I would not have picked it up where it not for some stranger on reddit that recommended it. Its surprisingly good considering its over 10 years old.  You feel the age a bit by the OOP buzz wording. However, there is stuff in here that I have not read elsewhere. For instance, inheritance is not about the IS-A relationships or sub-typing. Public inheritance is about substitutability, nothing more or nothing less. For instance, even though a circle is a subgroup of the group oval in the mathematical sense. Their interfaces probably still wont be substitutable. This is not a book for beginners though, its more of a “Best Practices” type book. I recommend it.

Cheers

Maths is Neat!

Wow, its been over a year since I posted anything here. This blog was meant to be only about C++ but that might change now. Two interesting books where published last year that I did not have time to read. They where Stroustrups new book “Programming: Principles and Practice Using C++” and Stepanovs “Elements of Programming”. The first is aimed at C++ beginners, the second at Mathematicians. Stepanov is the genius that invented STL.

That brings me to the point of this post. I always wanted to learn more maths but somehow never got around to studying it seriously. There was always something else. As a programmer you can make a career without any maths knowledge at all. I would guess 95% of programming jobs don’t require any maths. But then again 95% of programming jobs are quite boring. Its seems to me all the interesting programming is in maths heavy areas. For instance 3D programming is heavy with linear algebra, physics simulation is heavy with calculus. Algorithm and data structure design is also heavy with maths. That all interesting stuff.

So I bought a bunch of math book, put them in my backpack. Quit my job. Bought a one way ticket to South America. And for the  last year I have been living cheaply and studying hard. Who needs schools and teachers when you can teach yourself anything. It might sound crazy but its been great. I started studying pre-calculus, all that stuff I once knew and forgot. I moved on to study lots of linear algebra. I have also been studying a bit of calculus and about the math in games. I have also had time for sum fun and games, im in South America after all. By summer time I’ll be job hunting for a programming job that requires some of this math. I’ll try getting a 3D game programming job. I think that would be very interesting. My plan B is to go back to University and study maths. This stuff is interesting in its own right. I got nothing but respect and admiration for mathematicians and what they can do. Mathematics is the language of the universe after all. And what could be more interesting than that?

Maths is just great, especially in connection with programming. I hope I keep motivated to keep studying this for years to come.

Cheers!

Give it a name and its gone. Forget it and it moves back.

The most interesting explanation of rvalue references yet: 

http://codesynthesis.com/~boris/blog/2008/12/01/rvalue-references-affect-your-code/

Want to know whats the need for the std::move method? As I understand it, It’s needed because as soon as you give an rvalue reference a name it “becomes” an lvalue reference! This is probably done to make it harder for you to shoot yourself in the foot (as you would not want to use the name of a rvalue reference after its rvalue had already been moved away and the rvalue reference name was invalidated). So there needs to be a way to get back to an  rvalue reference from an lvalue somehow . That is what the move function is all about. std::move()simply converts any rvalue or lvalue to an rvalue reference, by stripping away its name.

Another look at Peterson

I had another read of Bartosz Milewski recent blog post about the different memory orderings of c++0x.

The most interesting thing about the blog post (that I missed last time) was actually the proposed Peterson locking algortithm that used some none default c++0x  memory orderings for the stores and loads of its atomic variables. It turns out that his proposed implementation is broken. Yupp, that’s right, even some of the experts on concurrency can get lost when straying from the straight and narrow road of  default sequential memory ordering. Exactly why the implementation is broken, and another implementation that actually works (by Dmitriy V’jukov) , is explained (or even proven) on Anthony Williams blog.

Anthony’s blog post was a very informative read. I probably read it ten times until I (think I) understood why one implementation was broken and the other wasnt. Lots of “ahaa – no wait a minute – oh OK, I think I see what you did there” moments. Reasoning about concurrency is allot like pealing an onion, there is always another layer underneath that you didn’t know about. 

(The problem as I understand it, was that the atomic variable “victim” was read and written by both threads,  and one needed to make sure that the writes that happen before on the first  thread where synchronized to the the reads that happen after on the second thread. Because the memory ordering for the write on the first thread was release only, and the read on the second thread was acquire only, the second thread could read an old value from it invalid cached even though the variable had been overwritten by the the first thread.)

Writing correct concurrent code is an interesting but complex task, and anyone claiming different are probably writing broken code but dont realize. If guess the thing to take from all this is that unless you can prove that its Okay to use a more relaxed memory model than the sequential consistent one, you really shouldn’t.

Keeping your memory in order

Here is a short description of the different memory ordering for concurrency in C++0x; Bartosz Milewski’s blogs port about C++ atomics and memory ordering.

The default for atomic variables is the familiar sequential ordering (memory_order_seq_cst), its also the slowest as it inserts lots of most memory fences that prohibit the CPU from reordering code as it sees fit. Between that and no ordering (memory_order_relaxed), there are also ordering that just insert fences on reads (memory_order_acquire), on writes (memory_order_release), and on both read and write (memory_order_acq_rel). Being able to choose a memory order other than the safe and slower default is a nice feature, and corresponds well with the C++ deign rationale of not having to pay for what you don’t use. The java volatile’s apparently works much like the c++0x atomic’s with the default sequential ordering, except java volatile’s are not atomic (there is a java library for that).

A little padding just makes you run faster

Herb Sutter posted an excellent new effective concurrency article; Measuring Parallel Performance: Optimizing a Concurrent Queue. Its surprising to see how much scalability can be gained by wasting space on padding, to make sure independent variables get to live on different cache lines.

Boost threads shows the way

What’s New in Boost Threads? Boost threads moves closer to C++0x. The interface looks nice. Threads and locks are now movable (can be stored and sorted in move aware containers). Thread’s constructor can bind the  functions arguments to the function. Lock can now lock multiples mutexes at once to avoid deadlocking (when succeeding to get one lock, but failing getting the other).

Also, here is a link to Anthony Williams blog (the author of boost threads and the upcoming C++ Concurrency in Action book).

Concurrent programming is so assembler

I have been reading Herb Sutters effective concurrency articles over the past last couple of weeks. I just finished reading the last one. I think he will compile them into a book when he has enough of them (there are about 17 of them at the moment). I’m sure I’ll buy the book once its out, as I got all his other book already.

What is clear is that concurrent programming has hit the mainstream, and its here to stay whether we like it to or not. Tomorrows PC’s might not have faster cores than today’s, but tomorrows PC’s might have a hundred times more cores than todays. Programming such a beast correctly, while keeping your sanity, will not be an easy task. Although an interesting subject, it comes abit of a surprise to me how low level concurrent programming today still is. For parallel C++ programs its not just the order of the code as you write it that matters, not even the compiled rearranged code order can be counted on, its the order of the code actually being executed on the cores that makes all the difference to the correctness of the program. All this rearranging behind the scenes by the compiler and the CPU is done to optimize sequential execution but it makes life very difficult for concurrent programmers. To sequential programs the order looks the same on all three levels, but to parallel programs the different ordering will be seen and this causes more potential bugs and more complex logic to get right. Its like a step down back into assembler programming to get the ordering and synchronization right, or even like a step one level below assembler code, down to the the actual machine code on the CPU.

The C++0x standard will address some of this, in that C++ will finally get a defined memory model. This will basically limit how the compiler and the CPU may rearrange the code you write. Then there will be a new keywords, atomic, that forces the compiler to play nice and synchronize reads and writes to a given variable between all the cores. Then there will be threads, thread local storage, locks, futures etc. But its all still very low level. Hopefully higher level libraries and patterns will emerge that make writing deadlock free, data race free, scalable programs easier.

In my humble opinion it’s a bit unfortunate that once a new thread is spawned, the language and compiler assumes that all data and functions are safe to be used without synchronization. You have to explicitly specify and single out the variables and functions that have to be synchronized, instead of the ones that don’t. Well that’s just my two cents. The switch in programming styles, patterns and tools to accommodate the concurrency revolution will sure make for an interesting decade. If anything is going to kill C++ it might be that parallell programs are just to complex to get right in a language that allowsshared state and imperative pprogramming styles. Then again, if that the case, then most of todays mainstream languages will get killed even worse.

Anyways, here are Sutter’s Effectice Concurrency articles.

Better functors with boost

Today I read about boost::bind

It replaces the old STL function/functor binders (although lamda functions are a better replacement when composing/nesting several layers of binders). The improvement over the old binders is that (by way of better deduction) the new binder is less picky about the function/functor type, it can bind to any function/functor with up to 9 arguments, and the order of the arguments can be rearranged when binding. Bind is now part of the TR1 of the C++0x standard.

Here is a article and tutorial about boost::bind by Björn Karlsson: “How the Boost Bind Library Can Improve Your C++ Programs”.  The article is from the book “Beyond the C++ Standard Library: An Introduction to Boost” I’m tempted to buy the book, although I first have to finish a couple of other C++ books im reading.

Torvalds uses horrible language

I have been waiting to see if some authority in the c++ community would bite and give a response to Linus Torvalds flame about how bad C++ is compared to C. Steven Dewhurst’s wrote a good response. But first, If you haven’t heard the gossip.. A while back Linus gave sound bites like:

“C++ is a horrible language. It’s made more horrible by the fact that a lot of substandard programmers use it. […] C++ leads to really really bad design choices. You invariably start using […] STL and Boost and other total and utter crap […] and anybody who tells me that STL and especially Boost are stable and portable is just so full of BS that it’s not even funny. [C++ leads to] inefficient abstracted programming models. […]  The only way to do good, efficient, and system-level and portable C++ […] is limiting your project to C [which] means that you get a lot of programmers that do actually understand low-level issues and don’t screw things up with any idiotic ‘object model’ crap.”

You can read the whole thread here if you like.

I find it a bit chocking that a highly respected programmer like Torvalds would write such statements about the most advanced language in common use. He did not give any references to back up his claims about C++’s inefficiency or importability. He gave very little context for his views, so its hard to see where he is coming from. I got the impression reading the thread that C++ is unpopular in the open source developer community.

It’s been a long time since C had any performance advantage over C++. A decade ago The inventor of STL, Alexander Stepanov, wrote a famous article and benchmark where he debunkes the myth of the abstraction penalty of C++ compared to C. You can read the whole paper here.

The current C++ standard has been out for a decade, and today’s main compilers and standard library implementations are very conformant. So I cant imagine portability being much of an issue today. Even assuming Boost has portability problems, the library is just another 3rd part library, so you cant really blame the C++ language for it.

As for bad code and programmers. The fact that C++ has data hiding and abstraction features, and has stronger type safety than C, makes it easier to write good code. Without using data hiding and abstractions, dependencies are stronger and their number greater than necessary. It seems rather anti-intellectual to prefer more detail over more abstraction. After all, math, the purest science of all is all about abstraction. Good code is correct code, first and foremost. And performance, although important, is a secondary concern. With C++ we get the best of both worlds. Powerful abstractions and high performance.

I see no benifit to C, other than it being a simple language to learn. C++ might be complex, but not needlessly so. Every feature (well almost) is there for good reason.

Anyways, finally, here is Steven Dewhurst’s (a well known C++ author) response to Linus comments about C++. I’d love to see Linus response. Maybe he could explain his reasoning better given another chance.