The reading list for scientific programmer

Question

I am working to become a scientific programmer. I have enough background in Math and Stat but rather lacking on programming background. I found it very hard to learn how to use a language for scientific programming because most of the reference for SP are close to trivial.

My work involves statistical/financial modelling and none with physics model. Currently, I use Python extensively with numpy and scipy. Done R/Mathematica. I know enough C/C++ to read code. No experience in Fortran.

I dont know if this is a good list of language for a scientific programmer. If this is, what is a good reading list for learning the syntax and design pattern of these languages in scientific settings.

Solution 1

At some stage you're going to need floating point arithmetic. It's hard to do it well, less hard to do it competently, and easy to do it badly. This paper is a must read:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Solution 2

I thoroughly recommend

Scientific and Engineering C++: An Introduction with Advanced Techniques and Examples by Barton and Nackman

Don't be put off by its age, it's excellent. Numerical Recipes in your favourite language (so long as it is C,C++ or Fortran) is compendious, and excellent for learning from, not always the best algorithms for each problem.

I also like

Parallel Scientific Computing in C++ and MPI: A Seamless Approach to Parallel Algorithms and their Implementation by Karniadakis

The sooner you start parallel computing the better.

Solution 3

My first suggestion is that you look at the top 5 universities for your specific field, look at what they're teaching and what the professors are using for research. That's how you can discover the relevant language/approach.

Also have a look at this stackoverflow question ("practices-for-programming-in-a-scientific-environment").

You're doing statistical/finance modeling? I use R in that field myself, and it is quickly becoming the standard for statistical analysis, especially in the social sciences, but in finance as well (see, for instance, http://rinfinance.com). Matlab is probably still more widely used in industry, but I have the sense that this may be changing. I would only fall back to C++ as a last resort if performance is a major factor.

Look at these related questions for help finding reading materials related to R:

• suitable-functional-language-for-scientific-statistical-computing
• books-for-learning-the-r-language
• what-can-be-done-in-r-that-cant-be-done-with-python-numpy-scipy
• r-for-finance-tutorials-resources

In terms of book recommendations related to statistics and finance, I still think that the best general option is David Ruppert's "Statistics and Finance" (you can find most of the R code here and the author's website has matlab code).

Lastly, if your scientific computing isn't statistical, then I actually think that Mathematica is the best tool. It seems to get very little mention amongst programmers, but it is the best tool for pure scientific research in my view. It has much better support for things like integration and partial differential equations that matlab. They have a nice list of books on the wolfram website.

Solution 4

In terms of languages, I think you have a good coverage. Python is great for experimentation and prototyping, Mathematica is good for helping with the theoretical stuff, and C/C++ are there if you need to do serious number crunching.

I might also suggest you develop an appreciation of an assembly language and also a functional language (such as Haskell), not really to use, but rather because of the effect they have on your programming skills and style, and of the concepts they bring home to you. They might also come in handy one day.

I would also consider it vital to learn about parallel programming (concurrent/distributed) as this is the only way to access the sort of computing power that sometimes is necessary for scientific problems. Exposure to functional programming would be quite helpful in this regard, whether or not you actually use a functional language to solve the problem.

Unfortunately I don't have much to suggest in the way of reading, but you may find The Scientist and Engineer's Guide to Digital Signal Processing helpful.

Solution 5

I'm a scientific programmer who just entered the field in the past 2 years. I'm into more biology and physics modeling, but I bet what you're looking for is pretty similar. While I was applying to jobs and internships there were two things that I didn't think would be that important to know, but caused me to end up missing out on opportunities. One was MATLAB, which has already been mentioned. The other was database design -- no matter what area of SP you're in, there's probably going to be a lot of data that has to be managed somehow.

The book Database Design for Mere Mortals by Michael Hernandez was recommended to me as being a good start and helped me out a lot in my preparation. I would also make sure you at least understand some basic SQL if you don't already.

View more solutions

Comments

[leon]

I have strong appreciation of Haskell :)

[Artelius]

In that case, learn assembly language. IMO the best way to do that is to write a toy kernel in assembly language, because you'll learn a million things besides.

[Artelius]

Oh yeah, and there's always The Art of Computer Programming (by Knuth)

[metal]

MC++D is a fantastic book, but it's not for C++ beginners like the OP, nor is it any more useful for specifically scientific applications than is the GoF's original Design Patterns. If you don't know how to write your own template classes and functions and partially specialize them, for instance, you'll need a firmer grounding in the language before picking up this book.

[metal]

You will learn a million things by learning assembler, but that's something like saying to learn biology, study physics first. Sure you'll learn a ton, but (a) not everyone needs to understand everything about how computers or software work deep down (though more general knowledge is a fine thing to have), and (b) there are other paths more immediately applicable to his field of inquiry that could also provide much insight.

[Georg Fritzsche]

I don't know about the specific needs of the OP, but for "design patterns in [some] scientific enviroments" its a valuable foundation imo. Some lab-teams here see it as the initial must-read, thats why i brought it up.

[Artelius]

@mlimber: it's a matter of opinion. Note that I used "suggest" and "IMO" about this issue. The OP should choose something that suits him.

[Graphics Noob]

Do not, under any circumstances, use Numerical Recipes to try to learn a programming language.

[High Performance Mark]

Shit, too late, by about 25 years. Oh, what a wasted life. And I stand by my comment that NR is an excellent text for learning scientific programming, which is about a lot more than a programming language.

[Totti]

Numerical Recipes was ok 25 years ago but it is a joke today.

[Totti]

The license... and the awful code and advise.

[Totti]

You might also be interested in "units of measure" in Microsoft's new F# programming language.

[gast128]

This book contains some esoteric C++ constructs; best fit for library design with C++ templates. It is a bit dated due to modern features as perfect forwarding and variadic templates. It does not contain information about numerical methods, modeling and software architecture.