These images are known as
favicons and you can make it anything you want, the thing to remember is that there are two ways you should present it. For compliant browsers you'll want a higher quality favicon, in PNG format or something similar, and modern browsers locate these using a <link> tag in the head of the document. So something like:
HTML
<link rel="icon" type="image/png" href="/images/favicon.png">
However, some older browsers aren't capable of understanding this and use a cruder system where they will simply check for /favicon.ico and if that's there they will display it (the newer browsers also do this for backwards compatibility, but it's best to use the standard way so you aren't just stuck with .ico). You will want to include both versions for best results.
EDIT: by the by, in case you're not sure, /favicon.ico implies a "favicon.ico" file which is in the "root" of the site, i.e.
http://site.com/favicon.ico and NOT
http://site.com/dir/favicon.ico