Every Bitcoin address contains both a public and a private key. The public key allows others to send bitcoins to your address and verifies the signature of the transaction to ensure everything is in order and finalizes the transaction. The private key, on the other hand, allows you to ‘unlock’ and spend your bitcoins. It does this by signing transactions, which tells the Bitcoin network that you are indeed the owner of the address in which the bitcoins are held and that the transaction is valid. Whoever holds the private key for a Bitcoin address is able to spend the bitcoins which that address holds, so in a very fitting analogy, your private key is essentially the key to the safe which is holding your bitcoins. You can also use the private key of an address to sign a message, verifying that you are the owner of the bitcoins held at any given address. This is all secured through mathematics, using asymmetric cryptography.