WordPress is a complete web application. It has a front and which displays the website. And it has a backend that can be access by an administrator or site owner via /wp-admin in order to publish new pages and change content on existing pages.
The best way to learn WordPress is to download the latest version of the software from WordPress.org (not to be confused with the hosted blogs on WordPress.com).
WordPress claims to have a five-minute easy install. But because WordPress is a PHP application that connects to a MySQL database, you need to create the MySQL database on your webhost before installing WordPress.
The official WordPress installation guide is on codex.wordpress.com. When installing any type of web application, make sure you don’t accidentally set the permissions to be 777. Here is more info about file permissions.
This tutorial will show you how to install WordPress 3.0 from start to finish.
Or here is a video tutorial on installing WordPress (although on an older version of WordPress 2.0).
codex.wordpress.com is also where you will find documentation and references.
If you run into any problem of questions, you can always ask on the WordPress forums.
WordPress is extensible. Once you become familiar with WordPress, you can add additional functionality by finding and installing WordPress Plugins from the Plugin Directory.