Yes-ish: While petroleum derivatives is one of the main exports, the state budget isn't (overly) reliant on it. The amount of the state budget that can be covered by petroleum money is limited by law to a few percent (currently 3% iirc), and changing this percentage requires a supermajority in the parlament. In short, this is partially in an effort to avoid Dutch Disease, and also ensuring that the income from petrol exports aren't spent all at once - instead it is saved up and has accumulated into the world's biggest pension fund.
And no, the export isn't very expensive. North Sea oil (Brent crude) is pretty easy to refine. Plus, since a lot of the well output is natural gas, this is exported to Continental Europe with no problem after Vovka decided somehow that Europe needs him. And as Bronzie pointed out: Extraction cost is relatively high, but export cost is dirt cheap.
Source: Am norwegian
EDIT: Also, it's interesting to check out this page from time to time. It's in norwegian, but I'm sure Google translate can handle it, to the extent the graphs don't already speak for themselves.