The reward goes to a lucky computer owner whose machine has found a special number called a "nonce." It's difficult to find the number, and the number that will work is different for each person trying to find it. I'll explain why in a moment. "Bitcoin mining" software, which is free and open-source, finds the number. It starts with a nonce of zero and runs an algorithm that takes two main inputs (as well as some others that we can ignore for now). The first main input is called a "hash" which is computed from all the new bitcoin transactions. The second input is the nonce. If the algorithm produces a small enough result, then the owner gets some bitcoin. This is because one of the new transactions adds the reward to the owner's bitcoin address. If the algorithm produces a result that is too large (nearly all of them are), then the software increases the nonce by one and tries again.
Remember I said that I'd explain why each person trying to find the number has to find a different one? This is because the "hash" of the new transactions depends on the data that represents those new transactions. Since two different people trying to find the number each have a different bitcoin address, the one new transaction that puts the reward into the owner's btc address will be different for each of them. That changes the hash, and therefore the nonce that will work.
Remember I said I'd explain what the public ledger is? It is commonly referred to as the "blockchain" because each time someone finds that special number (the "nonce"), they add a block of transactions to it. So the blockchain contains all the blocks, each of which contains a set of transactions that were made during a particular period of time (designed to be about ten minutes long). Bitcoin is transferred from one bitcoin address (or, account, if you like thinking of it as a ledger) to another by adding a transaction. The transactions already in the block chain must include one that puts bitcoins into the source address, and none that remove those bitcoins from it. Let's see how the ledger is handled to make sure no bad transactions get in.
I'll explain it with an analogy. Suppose someone says "Find me a number whose digits add up to exactly half of itself." We all know how to make sure that any proposed answer actually works, right? Bitcoin works the same way. If you add a transaction that moves bitcoins from an address that doesn't have any bitcoins in it, your transaction will be ignored by every (sane) bitcoin miner and therefore never get into a block. But lets suppose that an insane miner tries to add it. First, in order for his attempt to even get noticed, he'd have to find that special nonce. Once he did that, here's what would happen:
The new block would be broadcast on the Internet to other miners who would read the transactions in it. They'd see a transaction that attempted to spend bitcoins from an address that didn't have any bitcoins, and tag the new block as garbage or just throw it away, and then proceed with trying to find a nonce that worked for them, using only the valid transactions they've received. Every decent bitcoin client would also recognize this bad transaction and therefore ignore the entire block. In this way, the miners protect the integrity of the blockchain, and by finding nonces that work, they earn bitcoins, or "online rewards."