Issues When Using auto-py-to-exe

17 Nov 2018 General Python pyinstaller Hits

After helping many of people fix issues with auto-py-to-exe and PyInstaller, this post contains what I think you should understand and ways to fix issues you are having with auto-py-to-exe.


Introduction

A while ago I wrote an article on common issues when using auto-py-to-exe. In this post I aim to replace that post and explain more in greater depth.

This is going to be long, but hopefully your question can be answered in here. I'll go over installing auto-py-to-exe, the interfaces content, how to use it, debugging and common issues that I see appear time and time again.

Please do not feel offended if I have referred you to this page, it's just the case that many other people have asked the exact same question you are asking and you haven't been able to find the answers I or others have provided.

A Small Disclaimer

I cannot guarantee this post or any advice I give will fix all your issues with auto-py-to-exe and PyInstaller. This tool allows you to select options and give data to be fed into PyInstaller, that means that a lot of the errors that occur will be raised by PyInstaller, meaning I can't go and change how it works. Typically though, errors brought up in PyInstaller are due to incorrect configuration, thus this post aims to help you fix anything setup incorrectly.

If you have an issue that isn't mentioned in ths post, chuck it in the comments below and I will look into adding it if we can find a solution.

What is auto-py-to-exe?

auto-py-to-exe is a project I made in my free time to make packaging Python scripts to an executable/bundle easier. The interface uses Chromes app mode and a small Python server in the backend.

When the user presses "Convert .py to .exe", all the data in the interface is sent to PyInstaller and then the output from PyInstaller is sent back to the interface. auto-py-to-exe then cleans things up in the background when done (files/folders generated by PyInstaller) and allows you to find your project in the output folder in your current working directory.

Installation

To install auto-py-to-exe, make sure you have Python 2.7 or above installed. Ideally you will also want Chrome installed but this is not a must; it will open in your default browser and the only thing different will be the interface size. Execute the following in the command prompt / terminal to install auto-py-to-exe:

python -m pip install auto-py-to-exe

Wait for this to finish and make sure that the last line contains "Successfully installed auto-py-to-exe" dash some version. You might get a message about your version of pip but you can ignore it. Now to run the project, execute the following in the terminal:

auto-py-to-exe

This will open auto-py-to-exe and the output will be saved in output/ in where your current working directory is.

Possible Gevent Issue

There could be a situation when installing auto-py-to-exe that you are told you require Microsoft Visual C++ Build Tools. This occurs because the file that was downloaded from PyPI to be installed needs to be built and these tools will build it.

Generally the way to fix this would be to install the tools and run it again, but instead you can get a pre-built version. To do this, go to gevents PyPI page and click "Download files" on the right to bring you to the files that are on PyPI for this project. You will want to find the file that matches your setup and is a .whl file, for example, I am using Python 3.7 on Windows so I would look for cp37 in the "Python version" column, Wheel in the "File type" column and then in the files that match these two criteria, match my OS - Windows. This means the file I will want is gevent-1.3.7-cp37-cp37m-win_amd64.whl (currently gevent is at 1.3.7).

Layout of the Interface and What Things Do

At the top of the interface, you are asked for the script location. This is the script you call to run your project, so if your project is one script, put that here, if your project is more than one script, put the script that starts it here. The inputs outline will become blue if the file exists, otherwise it will be red.

Next you need to choose between one-file and one-directory. These are relatively similar but when using one-file with extra files like images or data files you need to modify your script to account for path changes. The difference between these methods and people ignoring the extra step required generally result in the bulk of complaints I get.

Under that is a selection of whether you want a console window to appear or not. Simply if you are developing a GUI application or something that doesn't need the console to appear, use "Window Based". If you do want the console to appear, keep "Console Based Selected" but keep in mind that the script needs to block somewhere otherwise execution will end and the terminal will disappear immediately; a work-around for this is to put input() at the end of the script to hold execution until input is provided.

After this you can select an icon for your executable. Make sure that the files are .ico and don't just rename a file to be a .ico, that is not how file types work.

Next there is a section to add files to your executable that are not Python files. Since PyInstaller doesn't find extra files like images you need to add them manually. When adding an entry, the file path/folder path needs to the in the box on the left and the destination on the right; the destination is the folder in the executable.

  • If you put a folder on the left, all files will be placed in the folder (and sub-directories keeping structure) specified on the right.
  • If you put a file on the left, this file will be put in the folder specified with the filename the same as the original.

If you're using one-file mode, you'll be provided a link to stackoverflow which tells you how to implement a small wrapper to find the files you added. I explain why you have to do this towards the bottom of this post including an example; if you do not follow this step errors will occur.

Under this is the advanced tab which holds all PyInstallers extra flags as well as where to save the project and setting the maximum recursion depth. Here are a few flags/options that can help with things:

  • Enable Recursion Limit: If a RecursionError occurs, try using this to set the depth to 5000.
  • -n: The name of the output folder/executable
  • --hidden-import: If the executable says a module is missing, make sure you have it installed and add it here; you can separate multiple modules by a comma.
  • --debug: Set this to all to help make debugging a lot easier
  • Extra Command Data: If you want to manually add arguments, paste them in this input.

The "Current Command" section tells you what would be called if you wanted to execute this in a terminal and the convert button is under this. After conversion you can clear the output or open the output folder where the executable/package was saved.

General Steps

The process of packaging a Python project to an executable is simple in most cases.

  1. Start auto-py-to-exe by executing auto-py-to-exe in the terminal
  2. Search for your script
  3. Decide on whether you want the output to be a single executable or a directory (a directory has many benefits discussed later)
  4. Decide if you want a console window or not
  5. Add an icon if you desire
  6. Add any extra files your project requires that aren't Python files (e.g. images, csv, databases, ...)
  7. Click the big blue button at the bottom to package the project
  8. Open the location of the executable/package and run it

If you have any issues with running your script, it may be due to incorrect configuration. This means you will now have to go though a debugging process to find what is occurring.

Debugging

To debug, make sure you have setup the fields in auto-py-to-exe like you had previously - all your extra files and other settings. Now go to the "Advanced" section and under the title "How to generate" put all in the box beside --debug. This will print out messages to the console to help you debug.

When debugging, make sure you select the "Console Based" button and using "One Directory" will help remove basic problems; you can switch back to "One File" when there are no more bugs and then fix the ones associated with "One File" mode. When all bugs are removed, you can go back to "Window Based" if you wish.

Re-package your project and open up the command prompt / terminal and cd to the directory where the executable / package was output to. If you are using one-directory cd into that directory. Now execute ./my_project.exe (substitute the names). This will run the executable file and any errors you were missing before will be output to the console and preserved.

Look at what these errors are saying and fix them. Now repackage like you just did and keep fixing bugs and repackaging until there are no more left and your project is working properly. After this, you can then move to one-file if you were using one-directory and want one-file, and repeat the process by still staring it with cmd and debugging on.

When everything is done and there are no more bugs in the modes you want, you can enable "Window Based" if you want and remove all from --debug.

Not every project can be converted to one-file. This is due to files being used by the application being lost every time the application starts/stops. To understand this more go down to "Demonstration of Files References After Packaging".

How to Fix Specific Issues

These are some of the issues I have come across or others have asked me about and ways to fix them.

PermissionError: [Errno 13] Permission denied: ...

This occurs because you are trying to modify files in a directory you do not have access to. A way to fix this is to run the script with admin privileges by opening cmd as admin and then running auto-py-to-exe one you have cd'ed to the directory you want the output to be in.

One reason this could occur is that you have opened cmd and am in System32. Make sure you do not accidentally modify files in this directory so make sure you are in a directory where you want to write files to when running auto-py-to-exe.

RecursionError

Go into the advanced tab and toggle the "Enable Recursion Limit" to on. This will set the recursion limit to 5000 and hopefully stop this from being raised.

AttributeError: module 'enum' has no attribute 'IntFlag'

Try executing python -m pip uninstall enum34 to stop enum conflicts. This is also a fix for the similar issue "Fatal Python error: Py_Initialize: unable to load the file system codec".

ModuleNotFoundError: No module named 'x'

Make sure the module is installed and then add it to --hidden-import in the advanced tab. If you need to add more than one, separate them by a comma (,).

For example, I have seen this occur with pandas._libs.tslib many of times before so add that to the input by --hidden-import. If you need to add more missing modules, you can use pandas._libs.tslib, my_missing_module.

ERR_CONNECTION_REFUSED

You need to be connected to the internet. This error appears when you cannot reach the servers.

Command “python setup.py egg_info” failed with error code 1

Update setuptools using pip install --upgrade setuptools.

FileNotFoundError

Make sure the file you are referencing actually exists, this error says that the file you are referencing doesn't exist. If you are using one-file mode, make sure you have followed the necessary steps.

General Questions Constantly Asked

How do I Convert All My Python Files?

Add the entry point in the script location. PyInstaller searches for imports to get the rest of the Python files required to run, so as long as you use the import keyword, your files will be added.

The Terminal Just Opens and Closes But There Are No Errors

If you double click to run your Python script, what happens? Does it open and close also? That means this tool has done it's job correctly and the script is finishing just like it should.

You most likely think the output should stay visible because you are always using IDLE or an IDE and that's what those tools do. Add a statement like input() at the end of your script to block execution and wait for you to press enter before closing.

'python'/'pip'/'auto-py-to-exe' is not recognised as an internal or external command, operable program or batch file.

This occurs because the path that these executables are located in is not on your path; thus cmd doesn't know where to look for them. You need to add these paths to the PATH environment variable, to find these execute the correct command to find the path.

import os, sys

# If 'python' is not recognised
print (os.path.dirname(sys.executable))

# If 'pip'/'auto-py-to-exe'/... is not recognised
print (os.path.dirname(sys.executable) + "\\Scripts")

Now go to my video on how to setup Pythons PIP and follow the instructions on how to add the path that was printed to the PATH environment variable.

If you are using something older than Windows 10, setting the PATH environment variable will be a bit different. If it is just one line, add a ; to the end and then paste the path after the ;.

The exe Doesn't Work on Another Computer

This may be an architecture issue. PyInstaller will build an executable using the architecture of the machine it was built with. This means if you are using a 32bit machine, it will create a 64bit executable. As with any other programs, you cannot run 64bit on 32bit but you can run 32bit on 64bit. Thus I recommend using 32bit python or compiling on a 32bit machine so it will work on both architectures.

What is the __pycache__ folder?

This is Python bytecode generated by Python. You can delete this if you wish.

My Antivirus Detected the exe as a Virus

This is your anti-virus vendors fault. Check out this. There is nothing I or PyInstaller can do to stop this.

Lots of Errors Appear in the Output

These warnings can be ignored in most cases. I have not currently found a situation where these are an issue, after-all, they are only warnings.

Additional Information and Explanations

Some things that are clear to people that use Python a lot are not always clear to new people. Here are some discussions of why and how things occur.

The Difference Between One Directory and One File

One directory puts all your files in one folder. You can easily add and remove files like you normally would in a folder. When your script modifies a file in it's folder, the file will still be modified when you run the script again.

One file mode is a bit different, instead of putting all the files in a folder, it puts them in something like a zip file which is contained in the end executable. When you run the executable, the files contained internally are unpacked to a new temporary directory. Due to one file unpacking on startup, it is a lot slower to start.

Also due to the files being unpacked to a new temporary directory on execution, your files that you modified added that were in the same directory as the executable will not be there on the next run because they are now in a different unknown folder. This means when creating files, you will want to use an absolute path that is not where the project files are (could use something like APPDATA on Windows).

Demonstration of Files References After Packaging

I made a small Python file that finds where it is located and then tries to print the contents of a file if it exists otherwise will create a new one.

import sys, os
if getattr(sys, 'frozen', False): # we are running in a bundle
    bundle_dir = sys._MEIPASS # This is where the files are unpacked to
else: # normal Python environment
    bundle_dir = os.path.dirname(os.path.abspath(__file__))

print ('Location : ' + bundle_dir) # Where the base file exists

file = bundle_dir + '\\test.txt'
print ('File is at: ' + os.path.abspath(file)) # Absolute path of target file
if os.path.isfile(file):
    with open(file, "r") as f:
        print ('Contents:\n' + f.read()) # Print contents of file if it exists
else:
    print ('Created a new file') # Create a file if it doesn't exist

with open(file, "a") as f:
    f.write('New Line\n') # Add a new line to see if is there next time

input() # Block to keep terminal alive
Running this un-packaged (.py)

The first time it will create a file beside the script. Every time it is run, the new line will be added and this will be shown in the output. This means When you modify a file, it will stay modified.

Running this when packaged using one directory

The first time it will create a file in the directory (which will be beside the .exe). Every time it is run, the new line will be added and this will be shown in the output. This means When you modify a file, it will stay modified.

Running this when packaged using one file

Every time this is run, it will create a new file in the temporary directory. This occurs because the .exe unpacks to a new directory every time it is run, so instead of finding old files (which can disappear any time because they are in the temp folder) you may as well use an absolute reference to somewhere else.

The one-file Resource Wrapper

As described in the section above, non-python files that are bundled into an executable when using one-file mode need care when being referenced due to how they are un-packaged.

In the interface, when one-file mode is selected and the additional files section is expanded, a note can be found that references a stackoverflow question regarding "Bundling data files with PyInstaller (--onefile)". The first answer provides a nice snippet of code:

import sys, os
def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")

    return os.path.join(base_path, relative_path)

When referencing files using relative references, instead of using open('folder/my-file.jpg'), you will want to use open(resource_path('folder/my-file.jpg')). This appends the relative path you provided to the current / extracted location to make an absolute file reference which is safe to use.

Basic Things To Remember

  • When using relative directories for file names, these are relative to the current working directory, not the location of the programs files. This is exactly the same as when you use a Python script normally. So depending on where you run the executable from, relative files my end up in different places than beside the executable.