How do I structure a file that is both a module and a CLI?
Keep the work in functions, put argument parsing in main(), and guard with if __name__ == "__main__": sys.exit(main()). Return codes from main then work for both the shell and the tests.
A comprehension building 2M items is slower than the equivalent loop. Expected?
Not usually, so look at what is inside. A comprehension that calls a method per item pays the attribute lookup every time; hoisting it out (append = out.append) or using a generator when you do not need the list is what actually moves the number.
A 300 MB JSON array does not fit comfortably in memory. Options?
Stream it. Either a pull parser like ijson, or, if the structure is a flat array of objects, read in chunks and use json.JSONDecoder().raw_decode to peel one object at a time. Memory then depends on the largest single object, not on the file.