Can someone explain this #Python import behavior

Baptiste Mispelon

Can someone explain this #Python import behavior?
I'm in a directory with 3 files:

a.py contains `A = 1; from b import *`
b.py contains `from a import *; A += 1`
c.py contains `from a import A; print(A)`

Can you guess and explain what happens when you run `python c.py`?

Alex Gómez

@bmispelon I feel like it would be either ImportError or printing 1, but knowing you I'm sure it's somehow the RecursionError

Baptiste Mispelon

@alexgmin Try it and see, then please explain the result to me, I genuinely don't understand what's going on

Baptiste Mispelon

@treyhunner Tagging you on this since it might qualify as a #Pythonoddity

Alex Gómez

@bmispelon Wtf

Chris May

@bmispelon My thought would be Python:
- runs c.py and immediately starts to import a.py
- sets `A` to 1
- then starts to import b.py
- module a(.py) is already in memory, so it bumps A to 2
- finished importing b and a
- returns to c.py and prints 2

Baptiste Mispelon

@_chrismay I think you're on to something. In my mental model Python would only "cache" an import after it's complete, but that doesn't appear correct.

Bonus questions for you then, what if `a.py` contains this: `A = 1; from b import *; A+=1`?
Or even this `A = 1; from b import *; A+=1;from b import *` ?

Eric Matthes

@bmispelon My initial guess was 2. c first imports a.A, which is 1. But the call to import from a loads a.py. That includes the call to import * from b, which imports from a. So at that point, A is 1. b then adds one to A, which sets A at 2. Then execution returns to c, with A at 2. So I think the value of A in c comes from b.

I tried to verify this in a pdb session, but stepping through at a low enough level to see this was bringing me into even lower level Python functions.

Jon Banafato

@bmispelon I *think* what's happening is basically sleight of hand (though I guessed wrong at first and would not advise that people use this behavior because it can confuse us). `a.A` gets reassigned when it imports `b.A` (via `b.*`).

Autoformatting these imports would cause an error, and obviously we want these files structured this way, so setting `__all__ = []` in `b.py` is the correct fix here.

Eric Matthes

@bmispelon Then I ran c.py in a VS Codium debugger session, watching A.

- A starts as undefined (everything does).
- After the first line of a.py, A is 1, but I think that VS Codium is actually reporting a.A.
- The import in a is hit, and A goes to undefined. I think VSC is showing b.A.
- b's import runs, and A is 1. I think that's b.A.
- The second line of b is run, and A is 2.
- Execution goes back to c, where the value of A is 2.

Baptiste Mispelon

@jonafato Interesting suggestion for a fix! What happens then if all the `from ... import *` are replaced by `from ... import A`?

Eric Matthes

@bmispelon Here's my VSCodium session:

Eric Matthes

@bmispelon Can you share the real-world motivation for this question at some point?

Baptiste Mispelon

@ehmatthes A very old Django project whose multiple settings files were importing from each other, leaving me very confused for a bit

I definitely would not recommend writing actual code that looks like this!

Eric Matthes

@bmispelon

> whose multiple settings files were importing from each other

You are not the only one who would be confused, please do not mention this in office hours

Timothée Mazzucotelli :python:

@bmispelon

Got it! Did it in my head then verified with an interpreter

There's nothing weird here. Python executes stuff sequentially, so:

- in from a import A
- in a: A = 1
- in a: from b import *
- in b: from a import * (so we have A = 1 in b)
- in b: A += 1 (so we have A = 2 in b)
- in a: finishing previous import, so we now have A = 2 in a
- in finishing previous import, so we now have A = 2 in C
- in print(A) -> 2!

Jon Banafato

@bmispelon That would result in the same original behavior, since `__all__` controls the import behavior of `*` but not of individual variables (though I think I have seen projects that allow you to turn that kind of thing into an error via name mangling or some other hack under the hood).

Timothée Mazzucotelli :python:

@bmispelon by the way I'm not sure to understand why the circular import works. I think Python has special handling for some cases where it's able to tell the circular import is "safe" somehow (like "a is almost finished, there's only * to import from b", meaning b can import from a again, and when b is finisehd a is updated again with any symbols declared in b). Tried to find an actual answer in the past but didn't find anything. Maybe should read the sources!

Timothée Mazzucotelli :python:

@bmispelon OK no it's much simpler, module a is simply partially initialized. By the time b imports it, it's not re-executed since it exists in sys.modules, and b imports every existing (yet) symbols within A. from a import A would work too.

Patrys

@bmispelon I think it’s a change from Python 3. Previously, cyclical imports would fail unless you only imported already defined symbols by name. It had its own WTFs as you could later redefine a symbol that was already exported. In Python 3 it instead keeps building each module in place until complete or deadlocked, so you get A = 1, block, A gets imported into b, A is increased, a gets unblocked, a imports A back. I’m sure there is a valid use case