Venture capitalists fear they may be one of the financial sector’s big losers from Brexit.

London is Europe’s venture capital center but the funds backing the bloc’s fintech and startup culture could lose billions of euros of funding once the U.K. officially leaves the European Union.

A main source of cash for VC funds in London comes from the European Investment Fund, an institution mainly backed by the EU that pumps money into venture capital funds and small nascent businesses.

The EIF has guaranteed a steady stream of funding over the past two decades to VC firms, 20% of which are based in the U.K.

According to Invest Europe, a lobby group representing venture capital and private-equity funds, the EIF committed €10.9 billion ($11.7 billion) to venture capital and small private-equity funds between 2012 and 2016. Of that amount, €2.8 billion, or nearly one quarter, went to the U.K. directly.

Venture capital is an important source of funding for startup and growing technology companies, many with high potential but facing higher risks with untested business ideas.

Unlike in the U.S., where the Harvard University endowment alone totals $35.7 billion and there are dozens of other endowments over $1 billion, Europe lacks the large-scale university endowments that invest in venture capital. Cambridge University’s endowment, the largest in Europe, amounts to nearly £6 billion ($7.3 billion).

“European endowments and pension funds allocate assets to venture capital, but it is mostly narrowed to the largest two or three funds” said Christian Hernandez, a partner at venture-capital firm White Star Capital. “The role of the EIF has therefore been critical to fill this gap.”

In 2015, the EIF provided U.K. startups with €655.8 million, which it estimated would generate an additional €2.87 billion in capital from other sources. At that point, it had supported 144 U.K. private-equity and venture funds and almost 30,000 small businesses, according to the EIF.

Such funding could disappear when the U.K. leaves the EU unless the U.K. strikes a special deal giving it access to EIF funds.

The loss of EIF cash would inflict pain on venture capital funds given the percentage of commitments to British-domiciled funds from the EIF, Mr. Hernandez said in an interview.

At the moment there is no clear guidance on what might replace EIF cash.

“The prospect of the U.K. no longer having those funds to invest creates a high level of uncertainty for the U.K. tech community,” said Romilly Dennys, executive director of the Coalition for a Digital Economy, a lobby for U.K. digital and tech startups.

The lobbying group is pushing the U.K. government to seek continued collaboration with the EIF and advocates that the commercial arm of the state-owned British Business Bank becomes a stand-alone institution with expanded funding before Brexit.

But the British Business Bank is currently less equipped than its EU counterpart. EIF has a staff of 369 compared with 10 in the commercial arm of the British Business Bank, the U.K. lobbying group said.

Like other financial firms, venture capital managers in London are assessing the consequences of the U.K. leaving the single market, said Invest Europe Chief Executive Michael Collins. “For some, this may comprise relocating to Europe’s other VC capitals, including Berlin and Helsinki, to ensure they will still have access to the money.”

Mr. Collins said that while London’s scale, combined with deep pools of talent and capital, isn’t easy to replicate, there are other cities with thriving startup scenes that are attractive to venture capital funds.

Berlin, for example, is strong in digital media, Helsinki in mobile gaming and Paris in e-commerce. The Association for Financial Markets in Europe, which represents firms in wholesale financial markets, said in a March report that the EU VC industry is concentrated around 10 members of the bloc.

Moreover, most VC firms employ small numbers compared with banks and aren’t trapped in long-term leases, making staff moves less onerous.

White Star Capital, which invests in the tech industry, for instance, is based out of London’s “Second Home,” a workspace in London’s hip Brick Lane neighborhood that boasts original Bauhaus furniture and weekly yoga classes.

While the loss of EIF funding could lead to a stronger VC presence in Paris, Berlin, and the Nordics, Mr. Hernandez said tax incentives and a tech ecosystem boosted by a handful of universities focusing on artificial intelligence and machine learning give the U.K.’s VC scene an advantage.

“While I believe the uncertainty of Brexit might slow it down and allow others to benefit,” said Mr. Hernandez, “I am hopeful that the government will realize the asset the digital economy has been thus far and continue to invest in incentives that support funding.”