• Recovery.elf and Recovery Mode - Recovery.elf is nothing more than a file that will initiate "Recovery mode". Recovery mode is no more than a simple application that will let you to use the USB, and execute a program under ms0:/PSP/GAME/UPDATE/EBOOT.PBP, that could be, for example, a Sony updater or the 1.50 -> 1.00 downdater by keeping R pressed while starting the psp.
  
• 'Flashing' - Writing to the PSPs Internal memory which holds the firmware information. If you mess up any of the firmware files your PSP could be a brick, although some can be saved by using recovery mode.
• Custom Firmware - Firmwares designed by homebrew coders. They aren't completely new, and only offer simple modifications of firmware 1.5 at the moment. Usually these come in the form of new or replaced "modules" or PRX's that make up the PSP's firmware.
  
• Overclocking - Setting the PSPs CPU clock speed to 333 Mhz. This cannot damage your PSP, although will use more battery life, and has been included in many Custom Firmware mods in order for the PSP to run faster.
• XMB - The PSPs "homepage". It's the menu which houses game, music, video, settings etc.
  

If you are incredibly unsure on how to use custom firmware, I would go with Dark_Alex's original release for starters. This granddaddy of Custom Firmwares and has a great Readme. This Custom Firmware features Recovery mode, can execute 1.0 eboots, can hide corrupt icons and can execute any homebrew program you wish on startup.

Download: [Dark_Alex Custom Firmware POC]
View: [Forum Release Thread]

• Zettablade's Custom Firmware All-In-One Mod
  

Has the ability to take screenshots using the note key, an automatic USB function (just plug the USB cable in and it works), and overclocks your PSP's CPU to 333Mhz.

Update: Zettablade has released a new version of his AIO firmware mod today. It includes bug fixes and a config file so that you can disable any of it's features.

Download: [Zettablade's Custom Firmware AIO Mod v0.2]
View: [Forum release thread]

• Harleyg's Custom Firmware mod v0.2
  

Harleyg's Custom Firmware is probably the most feature-packed yet. It has a much nicer looking recovery screen, includes options the option to both change the clock speed and remove the custom firmware completely. Homebrew can be started by holding the L-Trigger and much much more. A full list of features can be found here.

Download: [Harleyg's Custom Firmware mod v0.2]

• PSP Audio Recorder Beta
  

This is one of the older custom firmwares, before the release of Dark_Alex's proof of concept. The mod made by Moonlight It allows you to record audio from a UMD or homebrew application using L+R+Circle to begin recording and L+R+Square to stop recording, saving all recorded sound to PSP/Music in .wav format. This has a few bugs including sound lag whilst in-game, but is decent for those wanting to record some of their game.

It saves the recording as a WAV file in PSP/Music.

Download: [PSP Audio Recorder Beta]

• Slasher's MP3 Plugin
  

This allows you to listen to your MP3 from anywhere in the PSPs XMB using hotkeys assigned by Slashers MP3 Plugin. Currently supports up to 100 songs that you may have in the ms0:/PSP/MUSIC/ folder.

Controls -
LTRIGG + LEFT = Switch song(to the left)
LTRIGG + RIGHT = Switch song(to the right)
LTRIGG + SQUARE = Stop current MP3
LTRIGG + CROSS = Pause/Unpause

Download: [Slashers MP3 Plugin]