"Free memory" is kind of confusingly named. Your real "free" or usable, available memory is unused memory (aka "free") + cached. So according to your output there you have 126M "free" + 1274M cached, or 1400M of available memory.
Cached memory is actually better than free, what that means is that the libraries, files, etc are already in memory, just like query caching in mysql. If you want to use it again, its there and it wont try to load from the disk again. If you dont, then it just dumps it and gives you the memory from the cache.
Swap is going to get used for what are called "anonymous mappings". That is for data that has no associated files on the disk (like the mysql query cache for example). This is one of the ways that the kernel cleans up after itself. So even if you had 64G of ram, you still might end up with a few megs in the cache just from those anonymous mappings. Its nothing to be concerned about.
Last but not least, 64-bit systems will consume a lot more memory than 32-bit systems. So I always double the amount of ram I use on 64-bit boxes over the 32-bit ones.