Book collecting began around the time of the destruction of monasteries by the likes of Henry VIII. And those that were able to do so, began collecting the works that were previously stored by the monks as their own private collections.
Rarity of a particular book will not make it worth collecting it as it might be one of those books that have no particular intrinsic value, for various reasons. Unless it was particularly special to an individual in history, or was autographed by someone, though this would need verification and authentication as forgery is “easy” to do. Sometimes, a book that is not of any particular printing, or importance is valuable merely because it can be traced and is known to have been the possession of some important figure in history, and if such is the case,then it can be extremely valuable.
Book collectors can vary, from really serious people that would want to collect such works as the GUTENBERG Bible or Shakespeare’s First Folio… but the average collector would focus on the first editions of a particular author or some similar type of characteristic. In fact, the first editions of some works can fetch as much as the down payment on a house, though these are few and far between. And as most of these types of books are held by libraries and universities, they are unlikely to come onto the market.
A distinction should also be made between book collecting of modern works and Antiquarian Book collecting. This sort of item that collectors would be interested in is essentially prior to 1900. Prior to about 1455 all European books are hand written and so these items are extremely valuable, and scarce.
So, book collecting can and is a very interesting pursuit. A lot of value is ascribed to these works and people the world over follow this hobby/lifestyle.