If you have a photo-realistic image and use appropriate compression, they usually are. However, if you have a file like this one with only a few colors, then jpg has less or even no advantage. Also, the smaller the image dimensions, the smaller the file size advantage of jpgs.
The best way to gain experience of when jpgs work better and when to prefer png is to save both formats and compare them in terms of file size and image quality. It's also worth trying different compression levels to find the sweet spot between file size and quality. For images with larger dimensions, you can usually compress more without much loss of quality compared to images with smaller dimensions.
Brokenshield While this is good advice, and I appreciate the input, I would assume 17.2 kB vs 78 kB shouldn't make the site load as slow as it is though, no?
It depends on your hosting. In any case, 17.2 KB should load about 4-5 times faster than 78 KB. If that means 9 seconds vs. 2 seconds, you'll feel the pain, if it's 0.9 seconds vs. 0.2 seconds, it's obviously less of a issue. But even in the latter case, you should expect some penalization from Google, so it is always advisable to give some care to the images.