If you play RPGs and MMOs on your mobile, you’ve probably thought more than once that certain tasks are a real pain. Collect resources, train troops, repeat the same campaign over and over again…all of this lends itself greatly to automation. The good news is that, with the right tools, you can set up your own screen automations and let your phone do the dirty work for you while you do the fun.
In this article we are going to split hairs: You will see solutions with Python, ADB and scrcpy, autoclickers, automation apps like Macrodroid, details for RPG Maker and GameMaker Studio, and even how to deal with slider captchas. It is not a theoretical guide, but a practical review of what the community is already using to automate mobile RPGs, with its limitations, tricks and real problems (such as in-game text recognition).
Automate mobile RPGs with Python, ADB and scrcpy
One of the most powerful forms of create screen automations for RPG games from mobile is to use an Android phone connected by USB to the computer and run a bot from the PC. The idea is simple: the mobile phone shows the game, the computer sees the screen in real time and sends simulated touches based on what it detects in the image.
A typical case is that of Last Z Survival Shooter style games, where you have to upgrade buildings, move units around the map, collect resources and train troops. These are all routine tasks that fit perfectly into a script. Using Python (for example, with version 3.13.5), OpenCV and scrcpy, you can put together something like this:
1. You connect a spare Android phone to the laptop via USB, you activate ADB debugging and make sure the PC recognizes it. This will be your “bot device,” so you don’t lock your main phone or risk being left without a phone if the game crashes.
2. You use scrcpy to view and control the mobile screen from the computer. Scrcpy makes a very light streaming of the screen and also allows mouse and keyboard events to be sent as if they were touches. It is ideal for Python to “see” what is happening and act accordingly.
3. With OpenCV you capture periodic screenshots and locate key images (buttons, resource icons, menus). The bot can, for example, detect the button to collect resources, press it, go to the next building, train units and repeat the cycle in a loop.
With this approach you can even switch accounts automatically: the script closes the app, returns to the mobile desktop, touches specific coordinates to open the next account or game, waits for it to load and continues with the proposed sequence of actions.
The big problem with this scheme is that everything depends on screenshots and fixed coordinates. This implies several important limitations:
- Not portable between devices: If you change mobile phones or use an emulator, the resolution, pixel density and, therefore, the exact positions of the buttons change.
- Any interface changes in the game (moved button, new icon, temporary banner) may break the detection algorithm.
- You only see what is in the image– If you need to read complex statuses (number of troops, error messages, etc.), you depend on OCR or other tricks.
Still, for repetitive actions like collect, collect rewards or chain workoutsthis type of bot works surprisingly well as long as you keep the phone, resolution, and interface constant.
Limitations of OCR and in-game text reading
When you try to go a step further and make the bot “understand” the game state, you inevitably end up crashing with OCR. Tools like Tesseract They are a classic for reading text in images, but in mobile games they often return mediocre results: mixed letters, wrong numbers or directly almost unreadable garbage output.
This is an important barrier in complex RPGs, because to make good decisions you need to know what buildings are ready, if you have uncollected troops, if the campaign is available or if there is a resource error. Without reliably reading the texts and counters, your bot goes almost blind.
Some factors that cause OCR to work so poorly on mobile phones are quite obvious, but they have a lot of influence:
- Decorative and bordered fontsvery common in fantasy RPGs, difficult to segment for OCR.
- Background with effects or patterns (stones, metal, fire, etc.) on which the text is superimposed.
- Resolution and scaling: If you crop small areas of the screen, the characters become too small or blurry.
- Compression and artifacts by passing the image through scrcpy, capturing it or saving it.
To try to improve readability, it is common apply preprocessing with OpenCV: convert to grayscale, increase contrast, apply adaptive threshold, dilations or erosions to make letters bigger… Even with that, the result is usually irregular: sometimes it nails the text and other times it generates tons of errors.
In practice, many bot developers end up opting for a mix of image detection (icons, colors, shapes) and OCR very limited to specific areas where they know the typography is cleaner. In mobile RPGs and MMOs it is better to rely on more robust visual elements (for example, a truck icon ready to send) than on long texts.
Overcome slider captchas in repetitive campaigns
Another point where automations collide with reality is with the internal game captchas. A very common example in old RPGs on iOS and Android, such as Kingdoms of Camelot: Battle for The North, is the slider-type captcha: a puzzle piece that you have to drag to a space in an image.
The mechanics are simple: the same image always appears with a fixed gap and a piece moved a few pixels to one side. The player only has to swipe until the piece roughly fits in the gap, making the captcha very easy for a human but a little more cumbersome to automate.
If you are farming a campaign that you must repeat infinite times to get loot with drop probabilitythis captcha becomes the only serious obstacle to leaving a bot working for hours and hours. To automate it, you have two main ways:
- Visual approach with OpenCV: you capture the screen when the captcha appears, locate the hole in the puzzle in the image (which is usually always in the same area) and calculate the distance between the original position of the piece and the hole; then you simulate a swipe with that distance.
- Use of specialized AI models: train or use a model that detects the optimal position of the slider from the entire image. It is more powerful, but much more complex to assemble just for a specific game.
The key is that, always being the same illustration and varying the position of the piece a little, it can be solved looking for correlations in the texture of the piece with the part of the background where it should fit. In colloquial terms: you make the bot play “find the two equal parts” and, when it matches, you drag the slider to that point.
Of course, we must remember that automating captchas goes against the original intention of that mechanismwhich is avoiding bots. Many games can ban accounts if they detect identical movement patterns or use of automated tools, so it’s a good idea to be clear about the risk before you start messing around with this kind of thing.
Create and test touch games in GameMaker Studio
When we talk about automation in mobile RPGs, it is not only about “cheating” existing games, but also design our own titles thinking about touch control and how certain tasks could be automated legitimately (auto-battles, autocast skills, etc.). GameMaker Studio is one of the most used tools for this type of development.
The first thing to be clear about is that, when creating a game with GameMaker, you must decide from the beginning what the target platform will be: mobile or desktop, and within mobile, if you are going to target Android, iOS, Windows Phone or Tizen. It is not at all advisable to make the entire game thinking about Windows and, in the end, simply export it to another system.
Many developers have experienced firsthand the typical “I’ve already finished it, now I’m exporting it to Windows Phone, which is also Windows and won’t cause any problems.” Then come the weird bugs, incompatibilities, and hardware issues that are hell to debug with the finished product. The practical advice is very clear: test on the target mobile phone from minute one and do test builds frequently.
For touch control, GameMaker initially relies on the mouse eventswhich work almost the same as on PC. A tap on the screen is interpreted as a left mouse click. However, there are important differences:
- Middle button events don’t work (Middle, Middle Pressed, Middle Released) or the wheel (Mouse wheel up/down).
- Events like Mouse enter and Mouse leave They don’t fire either, because on mobile phones there is no cursor.
- He right button is simulated as double tap. There is no double click event, so, if you need it, you must program a “two taps in a row” logic or use the function device_mouse_dbclick_enable() to activate or deactivate this behavior.
In addition to events, there are mouse-related GML functions. Here it pays to use the device_* functions mobile-specific, instead of generic variables like mouse_x and mouse_y. For example, device_mouse_x() and device_mouse_y() They return the position of the tap in the room, but only when a tap has occurred. They are functions, not variables, precisely to better adapt to the nature of touch devices.
Virtual keys and efficient touch control
One of the typical mistakes when creating an action or platform game for mobile with GameMaker is try to use mouse events for the entire control. For example, create a “button” object and react to the mouse pressed to move the character or jump. It works, but the response is often slow and inaccurate, which ruins the gameplay.
The solution recommended by experienced developers themselves is to rely on virtual keys. The idea is simple: your game logic still reacts to keyboard keys (arrows, space, etc.), but on mobile you define screen areas that simulate those keys when the player touches them.
So you can have a single code for PC and mobile: in Windows you control with a physical keyboardand on Android or iOS you draw a virtual pad (left, right, jump, action) that internally maps to the same keys. The game responds just as quickly, because it still uses the keyboard-based input system, but the user interface is touch-based.
This approach is very useful for action RPGs, platform or exploration games where you need more precision and speed than what simple mouse taps provide. Incidentally, it simplifies cross-platform compatibility, because you don’t have to duplicate all the control logic.
“Back” button and multitouch on mobiles
Something that any mobile RPG developer cannot ignore is the physical or virtual “Back” button on Android (also present in Tizen and old Windows Phone). Many design guides practically consider it mandatory: users expect it to go to the previous menu, pause the game or exit to the main screen.
GameMaker makes it easy: you can map that button to the Backspace keycreating an event with the
Another key feature of mobile devices is the multitouch. You can rudimentarily simulate it with virtual keys and fixed zones (for example, a button on the left and one on the right, both pressed at the same time), but GameMaker offers a set of specific functions to handle up to five simultaneous touches:
- device_mouse_check_button– Checks if a given finger is tapping the screen.
- device_mouse_check_button_pressed– Detects the exact moment when the touch starts.
- device_mouse_check_button_released: Activates when that finger stops playing.
- device_mouse_x / device_mouse_y: X and Y coordinates of the touch within the room.
- device_mouse_x_to_gui / device_mouse_y_to_gui: X and Y position of the touch in the GUI (real screen), useful if you use views that move or scale.
The first parameter of these functions is an index from 0 to 4 to represent each finger. With a simple for loop you can check, in each frame, which fingers are active and whereand from there manage combinations of gestures (for example, moving the character with the left thumb and launching skills with the right).
For mobile RPGs with slightly more advanced controls, it is advisable to rely on these APIs, since They give much more flexibility than a single global tap and they allow you to design control patterns that are also more predictable if, in the future, you want to automate parts of the gameplay.
Using the accelerometer in mobile games
Another very interesting sensor for RPGs and exploration games is the accelerometer. Unlike the mouse/touch, which only gives X and Y coordinates on a surface, the accelerometer offers data on the inclination of the device in three-dimensional space: X, Y and Z axes.
In GameMaker you access this information with three simple functions: device_get_tilt_x(), device_get_tilt_y() and device_get_tilt_z(). They return values between -1 and 1, where the extremes correspond to approximate inclinations of 90 degrees to one side or the other.
The typical way to use it is link the movement of an object to the tilt of the mobile. For example, in a horizontal game you could do something like:
if display_get_orientation() = display_landscape {
x += sign(device_get_tilt_y());
} else {
x += sign(device_get_tilt_x());
}
This snippet checks if the screen is in landscape format and based on that chooses which accelerometer axis to use. With a simple sign() you convert the tilt into a discrete movement to the left or right. In an RPG you could use it for mini-games, puzzles or special sections where the player must balance an object, avoid obstacles, etc.
Beyond design, understanding these sensors well also helps envision how the game could be automated. If your logic is well structured (for example, clearly separating the input, logic, and rendering layers), you could later simulate tilts or touches from an external script in a controlled and safe way.
Automate tasks with Macrodroid and IFTTT on Android
Not everyone wants to build an advanced bot with Python and image recognition. In many cases, to automate parts of the game or the environment where you run it, it is enough to generalist automation apps like IFTTT or Macrodroid. Android, in this sense, is a very grateful field because it allows you to automate almost anything.
Historically, the “king” of this type of automation has been IFTTTbut in recent years Macrodroid has gained a huge niche and is especially interesting because intuitive it is to create complex macros without knowing how to program. It has already accumulated more than 10 million downloads and a very high average score on Google Play, so the community supports it.
The free version of Macrodroid includes advertising and limits the number of macros you can have active, but to start it is usually enough. If you are hooked on the app and need more, the paid version is a single payment and unlocks all limitations without monthly fees.
When opening Macrodroid for the first time, the number of options can be a bit overwhelming, but in reality its operation is based on three very clear pillars: triggers, actions and constraints. The mental flow is as follows:
- Triggers: events that the mobile detects and that trigger the macro (for example, battery at 40%, connection to a certain WiFi, opening a specific app or touching a notification).
- Actions– What the system will do when the trigger is met (turn off Bluetooth, launch an app, activate sleep mode, etc.).
- Restrictions: conditions under which the macro No should run, even if the trigger is activated (for example, only apply if you are not playing, or if the screen is off).
A classic example, outside of the gaming world, would be: “when the battery reaches 40% (trigger), turn off Bluetooth and mobile data (actions)”. If you want this to not happen while you are using an online RPG, you could add as a restriction that “the game app is not in the foreground.”
In the context of RPGs, you can use Macrodroid to automate the game environment: increase brightness and deactivate saving mode when you open your favorite MMO, silence notifications so as not to interrupt games, activate do not disturb mode at night except for alarms, etc. It is not a bot that plays for you, but it is a perfect ally to prepare your phone every time you sit down to play.
Automations and touch control in RPG Maker MV
If your RPG is made in RPG Maker MV and you want it to work well on mobile (or even be able to be played and automated comfortably), you must know the limitations of the engine on Android. By default, when you export an RPG Maker MV game to Android, control is based on mouse click/tap to move the character and manage the menus.
There are some details that change compared to the desktop version:
- For open or close menus it is necessary to do double tap on the screen.
- There is no built-in virtual pad by default; The character is guided by essentially touching the area you want him to go to.
If you’re looking for the control you see in most mobile RPGs (an on-screen directional pad and perhaps an action button or two), you need to turn to specific plugins. One that is often mentioned for this purpose is MBS – Mobile DirPad & Action Buttonwhich adds a direction pad and an action button adapted to the touch environment.
The community has not yet fully exploited the export of MV projects to mobile, so These plugins may have bugs, lack of polish or incompatibilities. Still, they are a great foundation for building more natural touch control than simple click-to-move. If you think about future automations, having a stable virtual pad always in the same area of the screen also makes life much easier for any tapping bot.
Create games with AI and automatic RPG generation
In parallel to all of the above, development tools that They use AI to generate almost complete games from text. A representative example is what “AI Game Maker” type solutions offer, capable of creating RPGs, dungeon crawlers or arcade games from scratch based on a simple description in natural language.
The concept is quite groundbreaking: you write something like “a dark dungeon with orcs and treasures” or “a neon runner on Mars” and the AI engine is responsible for creating the map, enemies, basic story, visual assets and gameplay. It can work in both 2D and 3D, from pixel art to more realistic environments, and generates games ready to play, export and share in no time.
Among the characteristics that usually stand out are:
- Instant generation of dungeons and RPG adventureswith AI that acts almost like an automated dungeon master.
- Integrated asset libraryoften based on open resources such as those at opengameart.org.
- Cross-platform export: Android, iOS and web, without having to build your own engine from scratch.
- Low entry curve: aimed at users without programming or design knowledge.
These platforms go beyond the typical textual “AI dungeon”, since build complete playable experiences and allow you to modify or expand them with your own ideas. Although they are not specifically designed to automate games, they do open the door to a new approach: Design RPGs and dungeons with self-play logic built in from the beginningtaking advantage of the fact that AI can generate rules, events and battle systems that adapt to your requirements.
If you combine this type of tools with screen automation techniques, you can reach a very curious point: AI-generated games that are, in turn, partially played by botsfor example, for mass testing, loot balancing or detecting optimal paths in complicated dungeons.
With everything seen, it is clear that the current ecosystem offers many avenues to create screen automations for RPG games from mobile whether setting up visual bots with Python and OpenCV, using apps like Macrodroid to orchestrate the environment, designing your own RPGs in engines like GameMaker Studio and RPG Maker MV with well-thought-out touch controls, or even letting artificial intelligence generate your worlds and adventures; In the end, the key is to fully understand the limitations of each tool (unreliable OCR, interface changes, captchas, sensors) and rely on practical combinations of image, tactile events and game logic to get the mobile to do for you all those repetitive tasks that are so tiring in RPGs.
Start a new Thread