Libraries in Windows 7, 8, 8.1 and 10 are collections of folders. For example, the Music library includes the current user's My Music folder and the Public Music folder (music files available for all users on the device). Technically, a library is just a customizable view of multiple folders. You can add your own folders to default Libraries and you can create your own fully personalized Libraries. In Windows 8, 8.1 and 10, Libraries help to add custom folders to File History (aka Previous Versions) for file versioning and backup.
Libraries are displayed in the Navigation Pane (the left pane) of Windows/File Explorer by default in Windows 7 and 8. In Windows 8.1 and 10, you must right-click or tap and hold inside File Explorer's Navigation Pane and turn on the Show libraries option to reveal the default set of Libraries.
The main difference between Libraries in Windows 7 and Windows 8/8.1/10 is that the latter ones allow specifying a Public Save Location in addition to your own Save Location. You can also set a custom icon for non-default Libraries in Windows 8 and newer.
Modifying default Libraries in Windows 7, 8, 8.1 and 10
To modify a library, open Windows (File) Explorer by pressing Windows Key+E on your keyboard. Click Libraries in Navigation Pane.
In Windows 8, 8.1 and 10, most of the actions below can be carried out using commands in Library Tools, Manage tab of Ribbon, right after clicking on a Library.
In Libraries, right-click the Library you want to modify and select Properties.
To add a folder to the Library, click Include a folder (Windows 7) or Add (Windows 8/8.1 and 10) button.
In Windows 7 and 8, you can only add folders on hard drives and indexed folders on network drives. In Windows 8.1 and 10, you can also use folders on removable drives (USB sticks, SD cards, etc).
Find and open the folder you want to add to the Library, then click Include Folder button.
The folder is now included in the current Library.
Alternatively, you can add any folder to any Library from Windows (File) Explorer. Just right-click a folder, open the Include in library menu and choose the Library you want to add the folder to.
Removing a folder from a Library
If you want to delete a folder from a Library, click the Remove button in Library Properties window. This will not delete the folder from hard disk, it will just remove it from the Library. For that reason, Windows will not confirm the folder removal.
Alternatively, you can remove a folder by expanding a Library in Navigation Pane of Windows (File) Explorer. Right-click the folder you need to remove and click Remove location from library.
Restoring default folders for a default Library
In case you removed some default folder from a default Library and you cannot remember which one, click the Restore Defaults button. This will recover all default settings for the Library. Please remember that this will also remove all the non-default folders you might have added to the Library!
Creating and customizing your own Library in Windows
Windows 7, 8, 8.1 and 10 are not limited to the default Libraries - Documents, Music, Pictures, and Videos. You can add your own Libraries for anything you like: work documents, holiday trip plans, etc.
To create a new Library, either right-click or touch and hold some empty area in the Libraries folder, select New and then click Library.
In Windows 7, you can also click the New library button in the toolbar.
In Windows 8, 8.1 and 10, click the New item button on Home tab of Ribbon and then click Library.
Type a name for the new Library and press Enter key on your keyboard to confirm. You can rename any library later by clicking on it once and then pressing the F2 key on your keyboard. Alternatively, right-click on a Library and select Rename from the menu.
As you can guess, the new Library is empty. Double-click to open it and then click Include a folder button to select the first folder to be included in the Library.
As described above, find and select the folder you want to add to your new Library and click the Include Folder button.
Next, open Library properties by right-clicking on the Library and selecting Properties from the menu. Add other folders to the Library using Include a folder button.
In case you have more than one folder in a Library, you can select which folder will be the default location for saving files. For example, if you create a document and save it to your Library, it will be saved to the folder that is specified as save location.
The current save location has a check mark to the left of its folder name. To select some other folder as the default save location, click it and then click the Set save location button.
In Windows 8/8.1 and 10, you can also set a Public save location for other user accounts. If you don't specify one, both default save locations will be the same.
Select a folder and click Set public save location. The public save location will have a check mark with two persons to the left of its folder name.
To set a folder view (displayed columns, view type, etc) best suited for the library, use the Optimize this library for dropdown. For example, General Items is best suited for mixed content - documents, photos, videos. But if you are creating a music library, select Music to see all necessary columns such as Bit rate, Artist, Title, etc.
You can choose to display or hide the Library in Navigation Pane. Enable or disable the Shown in navigation pane attribute as you wish.
In Windows 8 and newer, you can change the icon of your own custom Library (the action is not allowed for default Libraries). To do that, click the Change library icon button.
Select a picture from the Change icon dialog and click OK.
Click OK or Apply to save changes.
Deleting a Library in Windows
To delete a custom or a default Library, right-click or touch and hold it, then select Delete.
Windows will then confirm removal of the Library. Click Yes to proceed. This will move the Library definition file to Recycle Bin, but all folders that were included in the Library will stay intact and no files will be removed. A library is just a view, remember? 
If you accidentally removed a Library you needed, you can recover it from Recycle Bin as described in the Work with files and folders in Windows article. Or you can undo the action with a simple keyboard shortcut - CTRL+Z.
Restoring default Libraries in Windows
In case you deleted one or more default Libraries and you cannot use Recycle Bin for some reason, right-click the Libraries item in Windows/File Explorer's Navigation Pane. Click Restore default libraries.