RuneApps forums

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Question - findsubimg

I'm trying to figure out how findsubimg works.

I have a needle

igneous_geodeImage.src = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABkAAAASCAMAAACZ8IWSAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAMAUExURQAAAAAAAionJy0qKjItLTIwMDUwMDcyMjoyMjo1Mjo1NTo3Nzw1NTw3NT03Nz46OkA6OkA8OkA8PEI+PEQ+PkRAPkVAQEZCQEZCQkdERElEQklEREtGREtGRk1GRk1HR05JR05JSVBJSVBLSVBLS1JNS1JNTVNOTVROTVROTlVQTlZQUFdRUVlRUFlRUVlTUVpTU1pUU1xUU1xUVFxWVF1WVl5XV15ZV2BZV2BZWWFaWWJaWmJcWmRcWmRdXGdgXgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIKv3PAAAAEAdFJOU////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////wBT9wclAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAAGHRFWHRTb2Z0d2FyZQBwYWludC5uZXQgNC4xLjVkR1hSAAABG0lEQVQoUz3Rf1+CMBAGcK4fYlRbKdpqwmSAIZSAgK2yeP+vCu8Qef7j+R4fxs3qhsCQ4bHrBgHg86VYSWGP1guALVW0/cjS9+j5cTASuGKzTVZUNaYqhct7QoEb7orq8GXMt8Gs/bcFkdXB3fxltf49Hv/+MXEcR6EvkFAWwtdxvEmStm1TShJrErh2/BDrbZpmWbbb5XmeoQGKzcIEB6nO87LcV/W+2C1J7p9cbD9xtij3dd00zcGTgoMFfOYKIQqc7mtjlFKedAFPQPIqpcTemB+lgkApPB39z+2SwPN8nA4CrTUafod2MBEE5z7CaMbOgm/ZTv+CJthwRqvrBVc6mTqMac4oD5e99cGbsacOZryji2CoG2+n607q35Yx09nR/wAAAABJRU5ErkJggg==';

I have a haystack:

var newItemSubRegion = a1lib.getregion(localItemX, localItemY, itemWidth, itemHeight);

And when I do:

a1lib.findsubimg(newItemSubRegion, igneous_geodeImage, null)

And it's not producing an empty list. It's always producing the same list.

Sorry if I'm being too nooby, I thought I had this working, but I don't.

DaStewie

Member

Registered: 2018-11-23

Posts: 23

Re: Question - findsubimg

You could try defining the needle as:

ImageData.fromBase64(function(i) {
		TelosReader.phaseImg = i
	},
	'iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAC30lEQVRYR+2UX0hTcRTHv3PTa1HmKtGk4YsWlSnON1suayqZ9ZCbisR8sVVIJkQMhJ6CsL9mVlJaIMUm/smIwIdAwXRao0CICMUHUYjSzW3NP1PvbpzfuNeb2Fvgyz0v93d+5/zOOfdzzu+nSk1OEAQuFqrQEgS1Bip+FaKQHl4OISqGY/ukk6z3EW3i/kZx1vuIOVRUQE/ve6RnZEqJXR8GUFluZjrZai5WYWpyUrLLFxsVJf+BjQqW77EC+lxuFBTk4/vYODtbX1+PaLUazQ23IdoW/R62979FKsCUn49QIJJEl5KCF44O6A+nY/TrN3xyu1FijhDpffcWNReq2Jr82pxd7Evy7HET7ty8wdZElOiJQmfoLImt+jKu1V2PtFMkIC+AjLwQxpXaq5iYmMCMxwOjIQfBxRDTjYYjiFXxLMCevTqMfB7FCs/D6/UiOyMd6vAK2tq78KixAc7uN8xPq9WCAw+LxYLyc1acKS6CJzC/VoD4F+RMM1BcdJIdJAI2mw39/X3QJWhZS6jY1d9ziIpSsWCFp07DkGtk/mlpqQgH/TAYjWhsbsGTpka8bnfA7/chHBbwsqMbOUdzJTJ/ERgbj8wAyVYuBsm7dkgJg95fbE8sgNpVXVPLUDfcu4uh4Y8YGHKhtLQU/LyPtXIhtAzr+Uuw2+2sbcODA3j4tBUOZzteOZz/boF80OQJKahcd3T24HlrCzo6O5F56CAG3V9wYP8+0MAaj+UxkiS5pkJUWCthLTPDZDKh1l6HohN52LaFi7SAhqXEbJGGUF7Aeptcz8rSS4NG1zQufgf0+mxGgOiIg0bxCo8bMegaYVTPllXg1v0HawRoNTXrQ5J2u3TV2P3WaNgDRbbExN2IoceIi8X09A8k7Yxjvku8gOmfsywY6bwgQJcQD00MB58/wAZNFJohLlrDHrRAcAEzc36o5H+7GWulAIWAQkAhoBBQCCgEFAIKAYXAphP4Az52Y2lqTMyvAAAAAElFTkSuQmCC'
);

That's what works for me

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Re: Question - findsubimg

What exactly is TelosReader.phaseImg? I'm not finding anything about that

DaStewie

Member

Registered: 2018-11-23

Posts: 23

Re: Question - findsubimg

Oh i just grabbed a part of my code as example

that'd be the variable you put in findsubimg.

		if (!img) img = a1lib.bindfullrs();
		if (!img) return null;
		
		var phaseImg = a1lib.findsubimg(img, TelosReader.phaseImg);

I'm not using a third parameter too, not sure if that matters. var phaseImg should give you a list.

Skillbert

Administrator

Registered: 2014-12-30

Posts: 1,061

Re: Question - findsubimg

Just a couple things to note here.

1.
The findsubimage function needs an 'ImageData' object, not a 'HTMLImageElement' object. As DaStewie pointer out, the easiest way to get that is with the ImageData.fromBase64 function (need imagedetect.js). This function is asynchronous which means that it doesn't return the result but you get the result in the callback function.

2.
The image in your first block of code contains an sRGB header (https://i.imgur.com/KZMlDEN.png). This header makes the image decode into different pixel values depending on the specific screen settings a capabilities of the user. This is a real nasty problem that can lead to a lot of "but it works on my computer" situations. I personally use a program called TweakPng to delete this header on every image that i use for detection.

3.
If i understand correctly you want to detect the geode image in the inventory using this code. I have real bad news, all inventory models are rendered in 3d and thus differ according to the users graphics settings and hardware. It is probably possible to detect this, but a lot more complicated and unreliable than what 'a1lib.findsubimg' is made for.

Anyway if you're still interested in developing an app you should join the discord server. It's a lot easier to go over code and such there. https://discord.gg/G3SbcS8

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Re: Question - findsubimg

My actual goal was I wanted to create something that logged when I picked up certain items so I can do some analysis on the data later. I was hoping this was possible with this tool

Skillbert

Administrator

Registered: 2014-12-30

Posts: 1,061

Re: Question - findsubimg

If you only plan to use it yourself you should be able to get it working by just searching for the inventory image. In that case you only need only need to fix your image with tweakpng and import it using that imagedata.frombase64 function.

you can also try tracking it using the runemetrics menu, there is a script for it you can use here https://runeapps.org/imagelibs/droptracker.js

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Re: Question - findsubimg

I wasn't aware runemetrics tracked everything you got from stuff like mining and such. I'll have to look into that

Skillbert

Administrator

Registered: 2014-12-30

Posts: 1,061

Re: Question - findsubimg

Might not now that i think of it. I was thinking about birds nests that you actually have to pick up.

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Re: Question - findsubimg

Sorry if I'm being a pain. I did what you suggested. This is my code right now:

// There are 4x7=28 inventory slots
for (y = 0; y < 7; y++)
{
	for(x = 0; x < 4; x++)
	{
		var itemIndex = (4*y)+x;
		var localItemX = itemTopX + (x*itemWidth) + (x*itemSepX);
		var localItemY = itemTopY + (y*itemHeight) + (y*itemSepY);
		var newItemSubRegion = a1lib.getregion(localItemX, localItemY, itemWidth, itemHeight); // This is the region of an individual inventory slot
		
		var newJSON = JSON.stringify(newItemSubRegion);
		var oldJSON;
		
		if (items[itemIndex] != null)
		{
			oldJSON = JSON.stringify(items[itemIndex]);
		}
		
		if ((items[itemIndex] != null) && (oldJSON != newJSON)) // checking if the inventory spots have changed
		{
			console.log('Item Index ' + itemIndex + ' changed');

			const promise = new Promise((resolve, reject) => {
				ImageData.fromBase64((i) => resolve(i), "iVBORw0KGgoAAAANSUhEUgAAAB0AAAAaCAMAAAB8SKeFAAAABGdBTUEAALGPC/xhBQAAAwBQTFRFAAAAAAACKiAXLSAXEEJOF1xsRzUkTjonTjoqUz4qWUItWkQwYEYwYEcyYkkyaU01fl5CHHKHJIylJI+pJJKtJ525KrDQLbTTLbXUMMLlMMTnMs7zN93/RP//hGJEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAprR/QwAAAQB0Uk5T////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////AFP3ByUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAAYdEVYdFNvZnR3YXJlAHBhaW50Lm5ldCA0LjEuNWRHWFIAAADuSURBVDhPbdLRUoMwFIRhEmJDYkO1tWpEeP+3THdPDsEi/w3tfsAwA105zCAc9soZRQTetM63b0aMr6ZpPT1GqnMvPVPFNefzW1XnnBCRSgLqpuECXNYVMd3WhFbVjX3l7Pl0TXXve++9yzkPIaSkdyZy80MrMJxA9X5w+V1BEg0ncGdS0pHZWtVkOvNZWfdWOOFiUR227tYmBI3ThB86W3u9o188Zs4/VZGcwPPzhwgTvQhP8zzLpjqOo9z5gvYK+6OIyu1Yl2WR7UmL+adyQEe6RV15r3xL/HJwfFK+3/qG9avDP9G211TRDlApD3yQqg57eOWyAAAAAElFTkSuQmCC")
			});
			
			promise.then((value) => {
				var result = a1lib.findsubimg(newItemSubRegion, value);
			
				if (!result.length)
				{
					console.log("Did not find needle in haystack");
				}
				else
				{
					console.log("Found needle in haystack");
					console.log(result);
				}
			});
		}
		
		items[itemIndex] = newItemSubRegion; // set the current item index to the subregion so we can determine if it changed in the next check
	}
}

And for some reason it's still not finding the needle in the haystack. newItemSubRegion is the haystack which is the inventory slot I'm checking

Skillbert

Administrator

Registered: 2014-12-30

Posts: 1,061

Re: Question - findsubimg

I gotto go now but here is a quick example of how i would do this.

ImageData.fromBase64(loaded,"iVBORw0KGgoAAAANSUhEUgAAAA8AAAAOCAIAAAB/6NG4AAABFUlEQVQoU5XQQUvCYBzH8Ydeh4cdBoPB4EGILh1kCCPGIkkcSYmIJstDGWpElzqF3SOkHeoaBOLB4SvQm74k///9nj1TOgXfi38/v8Em/HgmxAFVGn+qXifuOHbfYvOsbrg+gXC+Rqx1RI3yqeFVKH303n+UTja5pislO4Ni78lp3uqL0slGaZycRu/o/gWa+ktZm0F42H9GrLtD2e5T9mUkrweU07qzqk0kqtOV1vR41rpM6wFrDGQ04rrD4HuBdjVSWtFoFHwl/9QoG8BZlStKnE+XMnpIY128eYT2Pn4p+gmnNL+Z0jygv+Eo+6JDWbVWrncoB6E1UwRN38UO23mpxmBPpwNh+rW9QaaRpoXSSeG4vAVD3vp2wCNPlQAAAABJRU5ErkJggg==");

function loaded(buf){
	//captures the image but doesn't transfer it to the browser yet
	var imgref=a1lib.bindfullrs();
	//this function also works on imgrefs that are still in alt1 memory
	var locs=a1lib.findsubimg(imgref,buf);
	console.log(locs);
	var firstmatch=locs[0];
	if(!firstmatch){
		console.log("no match found");
		return;
	}
	//get the actual pixels of the small subregion we need
	var captbuffer=imgref.toData(firstmatch.x-10,firstmatch.y-10,30,30);
	//this is a debug function that dumps ImageData's to the DOM
	captbuffer.show();
}

In terms of program speed there is a hefty overhead per capture oparation so you usually capture a area around all the subregions that you need in one go. In opengl capture mode it also has to wait for rs to render the next frame so it will take 28 frames to run your code.

In this code it matches a crystal geode btw, to run it on other items you'd have to change the image string. Make sure to remove the aRGB header tho i'm not sure if that string you had earlier will work.

Last edited by Skillbert (2019-02-17 05:01:12)

onigumo1990

Member

Registered: 2019-02-16

Posts: 6

Re: Question - findsubimg

Just an update for everyone, I found my issue. It was the fact I was using for loops with an async function. When the function finally returned my for loops were done so it was constantly looking at the last inventory slot. I have it working now and it's printing out a log for me. My next step is to figure out how to save this log, but as far as this thread goes you all helped me a lot and I appreciate it. Thanks a ton!

Kes

Member

Registered: 2017-07-10

Posts: 20

Re: Question - findsubimg

@Skillbert,
As a suggestion based on this post, is it possible to add another forum for Alt1 3rd party app releases? So we can use this development forum to ask questions, help eachother and write tutorials?

Skillbert

Administrator

Registered: 2014-12-30

Posts: 1,061

Re: Question - findsubimg

Good idea, i've just split the two and added some tags.