I work for a large owner of Multifamily properties just to qualify myself.
Yes, there is a large "national databases" as you say, that are run by private companies that charge the apartment building to provide this information to them. However, most properties do make money on the application fee-- probably $5 or $10 bucks.
Not all properties have the same qualifying criteria, and there is no way to access previous reports run for other buildings.
In the case of the company I work for anyway, that is the way most institutional owners of real estate operate. Also, most instutional owners are funded by pension funds etc... and are in business to make money. As the previous poster who owns some rental properties noted, you can go to smaller (non institutional) companies and probably avoid some of these fees, but you may or may not get the same level of service and you could be moving in next to a mass murderer.
I would say, however, you can request them to waive the app fee and I bet 9 times out of 10 they will do it. They would much rather give you concessions up front then a reduction on the rent you pay every month.