A pretty well-balanced piece from Mike Rappaport:

First, the easiest way to pursue a Bleeding Heart Libertarian approach is to identify and seek to eliminate interferences in the market that operate to harm the poor.   There are many regulations that prevent the provision of services that would benefit the poor as well as other regulations that prevent the poor from being employed.  Many regulations do great harm to the poor by raising prices for goods and services that they could otherwise purchase at cheaper prices, such as regulations involving health care, health insurance, the environment, and zoning, to name just a few.  Other regulations also harm the poor by making it more difficult for them to enter a large number of trades from hair styling to driving taxis.  In addition, there are ineffective public schools in poor neighborhoods that the poor are largely forced to attend, which could otherwise be avoided through the use of vouchers.

Second, there are serious problems created by the social insurance programs that justify their elimination.  These programs, from retirement pensions, to medical care, to unemployment insurance, operate in the main not to benefit the poor, but to transfer funds to the politically powerful, such as the elderly or organized labor.  When one considers just how much is paid by lower income people in payroll taxes for these programs the harm is quite significant.