Germany’s persistent current account surplus reflects to a large extent low domestic private investment. We argue that two factors—the local fragmentation of Germany’s banking system and the role of local banks in local public finance—can help explain why investment is so low. Local public banks dominate lending to small and medium firms in many regions of Germany. At the same time, they are also important as lenders to local municipalities. Local banks in more indebted municipalities lend more to local government. This effect is particularly strong for local public banks, which are under the direct political control of municipalities. Local government debt yields very low returns, but because lending markets are geographically segmented, local banks can charge higher rates on their small and medium-sized enterprise (SME) customers to break even. Using a unique data set of more than 1m German firms over 2009-2016, we show that firms which depend on local banks pay higher interest rates on their loans if the local bank lends a lot to the local government. This considerably lowers firm investment. Hence, the reliance of municipalities on local banks for credit leads to the crowding out of private investment in geographically segmented banking markets.