It's hard to be sure without seeing a test page, but the problem here is probably down to the way you are using jQuery. In general, you can set a variable in a preloader extension and then use it in a load rule.
Preloader extensions execute immediately that the utag.js file is loaded in the browser. Depending on whether you are using a synchronous or async insert, and whether the utag.js is towards the top or the bottom of the page, this might well be before the DOM Ready event, and before the iframe in question has been rendered in the DOM. You need to check also that the jQuery library has loaded before you are executing this extension.
One alternative mechanism to get the same effect (without changing your html) would be load the SiteCatalyst tag on All Pages. Then create a JS Code extension scoped just to that tag, have this code in it:
var iFrameTaggeo = $("iframe[class='notag']").length;
if (iFrameTaggeo > 0) {
return false; // do not fire tag
}
If you have an extension that is scoped to a specific tag, and the extension returns false, the tag will not fire.