Does the mere presence of big banks affect macroeconomic outcomes? In this paper, we develop a theory of granularity (Gabaix, 2011) for the banking sector, introducing Bertrand competition and heterogeneous banks charging variable markups. Using this framework, we show conditions under which idiosyncratic shocks to bank lending can generate aggregate fluctuations in the credit supply when the banking sector is highly concentrated. We empirically assess the relevance of these granular effects in banking using a linked micro-macro dataset of more than 80 countries for the years 1995-2009. The banking sector for many countries is indeed granular, as the right tail of the bank size distribution follows a power law. We then demonstrate granular effects in the banking sector on macroeconomic outcomes. The presence of big banks measured by high market concentration is associated with a positive and significant relationship between bank-level credit growth and aggregate growth of credit or gross domestic product.