The actual file loaded is PACKAGENAME.m2, which should be on the load path and should contain a package named PACKAGENAME.
In order to accomplish this, several steps are performed (or bypassed, depending on the values of the optional arguments).
The current value of prefixPath is used to determine how to direct documentation hyperlinks; the link will be directed to the appropriate file if one is found by searching the trees referred to by prefixPath. Otherwise, all documentation hyperlinks are relative to positions within a single tree of directories, as describe by Layout.
It might be necessary to run installPackage twice if a package with the same name is already installed: the second installation will redirect the hyperlinks to the freshly installed documentation, because the files will have been installed by the first installation. This applies, for example, to those authors who are developing updates to packages already included with Macaulay2.
The files of the package are placed in subdirectories of the appropriate prefix directory as specified by Layout, depending on the value of the SeparateExec option: when it is false the files are all in share/Macaulay2/PACKAGENAME.m2, share/info/PACKAGENAME.info, lib/Macaulay2/PACKAGENAME/, share/Macaulay2/PACKAGENAME/, and share/doc/Macaulay2/PACKAGENAME/; when it is true the paths to the files are modified to reflect the type of your machine, e.g., common/share/Macaulay2/PACKAGENAME.m2, common/share/info/PACKAGENAME.info, x86_64-Linux-Ubuntu-18.04/lib/Macaulay2/PACKAGENAME/, common/share/Macaulay2/PACKAGENAME/, and common/share/doc/Macaulay2/PACKAGENAME/.
In addition, if no errors occurred during running the examples, then an empty file whose name is lib/Macaulay2/PACKAGENAME/.installed or x86_64-Linux-Ubuntu-18.04/lib/Macaulay2/PACKAGENAME/.installed is created, to signify that installation was completed.