Hey guys!
So, a few of us have been talking and poking around and thought it would be a good idea to post some info on the puzzle creator and the interesting things we've found. Not all of this information will be useful, some of it is just for fun discussion and exposition! This post is a work in progress.

**Puzzle Editor Files and Folder Structure**
First up, the folder structure.
steamapps/common/portal2/portal2_dlc2/ : contains the .vpk files and other files used for the program itself.
..../models/puzzlemaker : A very interesting thing here is that valve has included MANY .3ds (source files) for various models. Very useful to anyone that wants to modify the models or animated out-of-game!

steamapps/common/portal2/sdk_content/ : the good 'ol sdk_content folder is still being used
..../puzzles/ : contains a single devtest.p2c file. I haven't figured out how to open this yet.
..../maps/ : the good 'ol maps folder is used as an intermediary for compiling levels
..../maps/instances/p2editor/ : contains the new instances that the editor generates. Useful for anyone that wants to work in Hammer with the convenience of the new instances!

steamapps/common/portal2/portal2/puzzles/<long string of numbers>/ : this long string of numbers is YOU, or your steam ID, same thing. That's all we are anyway right? NO! I AM NOT A NUMBER, I AM A FREE MAN! Ahem. So this folder contains the .p2c files that the puzzled editor works with and .jpg's used for the preview. More on that in a bit.

steamapps/common/portal2/portal2/maps/puzzlemaker/<long string of numbers>/ : this folder contains the .bsp's that the puzzle generator creates. Note: you CAN run puzzle maker maps from the console by typing "map puzzlemaker/<long string of numbers>/<long string of numbers>" if you reeeeeally want to.

steamapps/common/portal2/portal2/maps/workshop/ : this folder contains folders containing the maps you've downloaded from the workshop. These folders are also named <long string of numbers> and each contain a single .bsp and it's preview .jpg

**The Build Process**
As I mentioned above, the PeTI editor works with .p2c files (see below for file format documentations). When you click build this is what happens:
1. .p2c file is converted to a .vmf placed in steamapps/common/portal2/sdk_content/maps/ . This file is called preview.vmf
2. the preview.vmf file is compiled using vbsp, vvis, and vrad as normal with these settings:

vbsp.exe: -entity_limit 1750
vvis.exe:
vrad.exe: -final -staticproppolys -staticproplighting -textureshadows -hdr
(thanks Robdon)

1. the preview.vmf is paced in [size]steamapps/common/portal2/[/size]portal2/maps/puzzlemaker/ . (NOT in the <long string of numbers> subfolder).
2. the new preview.bsp is loaded in-game!

When you publish a map the .vmf created is named <long string of numbers>.vmf instead of preview.vmf and is placed in your STEAMID sub-directory. Publishing your maps to "Hidden" will allow you to create unique .vmf's without people seeing them. This means you can then take the vmf into Hammer. You can of course, simply rename the preview.vmf if you would like. Whatever is easiest for you.
NOTE: I have noticed some <long string of numbers>.bsp files in my STEAMID subfolder of maps that I have NOT published. This means that under some unknown condition the map may be built or renamed with it's unique identifier rather than as preview.bsp

**.p2c File Format - It's voxel time!**
Here we will start to get technical. Please note this is incomplete and contains some "educated guesses"

The .p2c file format will look somewhat familiar to anyone that has looked at a .vmf in Notepad, or written a .vmt. It is a KeyValues file. KeyValues files are ASCII files based around pairs of, you guessed it, keyvalues: a property and it's setting. They also have groups delimited by {}.

All .p2c files start with something like this:

"portal2_puzzle"
{
   "AppID"      "644"
   "Version"      "12"
   "FileID"      "0x0000000000000000"
   "Timestamp_Created"      "0x000000004FAB46CA"
   "Timestamp_Modified"      "0x000000004FAB4743"
   "CompileTime"      "12.694000"
   "Title"      "hack_tester"
   "Description"      "(Provide a description of your creation)"
   "PreviewDirty"      "0"
   "ChamberSize"      "8 6 4"

"portal2_puzzle" : I'll let you guess what that means. This is always here and starts the file. Everything else in the file is grouped under it.

AppID : This is a SteamAppID. Portal 2's is 620. I assume 644 refers to the Puzzle Creator

Version : Assumedly the version number of the .p2c file format being used

FileID : Always 0x0000000000000000

Timestamp_Created : Assumedly the time the file was created in hex

Timestamp_Created : Assumedly the time the file was last modified, in hex.

CompileTime : Assumedly the amount of time it took for the last "build" process.

Title : This is where the name of your map is stored, it's what you type in the field when saving.

Description : The description you give your map.

PreviewDirty : This variable is used set to true if the editor needs to re-create the preview thumbnail of the map. It's only used by the editor internally - you won't notice much of it - (Djinndrache)

ChamberSize : This is the extents of your map, in voxels (xyz)

Next up is the voxels lump. First, a word about voxels. Voxels are sexy big-news next-gen of the last-gen but still next-gen. Crysis' awesome terrain? That's voxels. Basically, voxels are 3D pixels: volume-xel , get it? You may have also heard of luxel and texel. These are all types of "resolution elements", or resels (according to wikipedia).

"Voxel"
      {
         "Position"      "0 0 0"
         "Solid"      "0"
         "Portal0"      "0"
         "Portal1"      "0"
         "Portal2"      "0"
      }

"Voxel" : each voxel definition starts with the word voxel

Position : the point in space defining the center of the voxel. These are XYZ integer coordinates, each unit = 128 hammer units.

Solid : this is a boolean (0 or 1). I believe 0 = nothing here, 1 = something here. If so, that means that even the empty space is stored.

Portal0 : another boolean, 0 = non-portable, 1 = portable
Portal1 : another boolean, 0 = non-portable, 1 = portable
Portal2 : another boolean, 0 = non-portable, 1 = portable

There are always 3 Portal0-2 values. This means that the program stores a single floating cube as multiple voxels (in my test it was 4 voxels). Exactly how this works I do not know. I would have assumed that 1 voxel = 1 cube.

After the voxel section is the "Items" section. This is where the gameplay instances are.
For example:

      "Item"
      {
         "Index"      "3"
         "Type"      "ITEM_CUBE"
         "Deletable"      "1"
         "VoxelPos"      "2 4 0"
         "LocalPos"      "0 0 0"
         "Angles"      "0 0 0"
         "ITEM_PROPERTY_CUBE_TYPE"      "1"
         "ITEM_PROPERTY_DROPPER_ENABLED"      "1"
         "ITEM_PROPERTY_AUTO_DROP_CUBE"      "1"
         "ITEM_PROPERTY_AUTO_RESPAWN_CUBE"      "1"
         "Facing"      "0 0 0"
      }
      "Item"
      {
         "Index"      "4"
         "Type"      "ITEM_DROPPER_CUBE"
         "Deletable"      "1"
         "VoxelPos"      "2 4 3"
         "LocalPos"      "0 0 0"
         "Angles"      "0 180 180"
         "ITEM_PROPERTY_CONNECTION_COUNT"      "0"
         "ITEM_PROPERTY_CUBE_TYPE"      "1"
         "ITEM_PROPERTY_DROPPER_ENABLED"      "1"
         "ITEM_PROPERTY_DROPPER_FALL_STRAIGHT_DOWN"      "1"
         "ITEM_PROPERTY_AUTO_DROP_CUBE"      "1"
         "ITEM_PROPERTY_AUTO_RESPAWN_CUBE"      "1"
         "Facing"      "0 0 0"
      }

Here we see a cube and it's associated dropper. It's a companion cube! <3

Index : the first item in the file is 0 and they keep going up in number.

Type : a name such as ITEM_CUBE . Presumably there is a file somewhere indexing these to the instance .vmf's

Deletable : as you may know, you can't delete the doors or the big observation window. Those items have this set to "0". Setting it to 1 did not allow me to delete. Removing the items from the file manually crashed Portal 2 when I tried to load the file.

VoxelPos : assumedly the position of the item in voxel coordinates.

LocalPos : used for items, such as the Light Bridge, that can be placed at 32 unit increments. A light bridge at the bottom of the space gave the value "-0 -0 -0.375"

Angles : an angle value. This changes with the direction, in world space, that the item is facing. Make a cube, and place a light bridge on it, now move the light bridge to different sides and the angles value changes!

ITEM_PROPERTY_* : following angles there is a number of ITEM_PROPERTY_ values. These values depend on the item being placed.

Facing : not sure about this one, I haven't seen anything give a value beside "0 0 0"

Next is the Connections section:

      "Connection"
      {
         "Sender"      "3"
         "Receiver"      "4"
         "Type"      "CONNECTION_BOX_DROPPER"
      }

A connection is the simplest type of entry in a .p2c file. Each entry has the same three properties:

Sender : this is the index number of the item on the "front" of the connection
Receiver : this is the index number of the item on the "back" of the connection
Type : this is a name, the "type" of connection.

In the above example, the ITEM_CUBE is linked to the ITEM_CUBE_DROPPER with a CONNECTION_BOX_DROPPER

And thus ends the file!
Here is the overall format:

"portal2_puzzle"
{
   "AppID"      "644"
   "Version"      "12"
   "FileID"      "0x0000000000000000"
   "Timestamp_Created"      "<hex value>"
   "Timestamp_Modified"      "<hex value>"
   "CompileTime"      "<duration of last compile>"
   "Title"      "<map name>"
   "Description"      "<map description>"
   "PreviewDirty"      "0"
   "ChamberSize"      "<map extends in voxels>"
   "Voxels"
   {
      "Voxel"
      {
         "Position"      "0 0 0"
         "Solid"      "0"
         "Portal0"      "0"
         "Portal1"      "0"
         "Portal2"      "0"
      }
      [...]more voxels[...]
   }
   "Items"
   {
      "Item"
      {
         "Index"      "0"
         "Type"      "ITEM_ENTRY_DOOR"
         "Deletable"      "0"
         "VoxelPos"      "0 2 2"
         "LocalPos"      "0 0 0"
         "Angles"      "-90 -90 0"
         "Facing"      "0 0 0"
      }
      "Item"
      {
         "Index"      "1"
         "Type"      "ITEM_EXIT_DOOR"
         "Deletable"      "0"
         "VoxelPos"      "6 7 3"
         "LocalPos"      "0 0 0"
         "Angles"      "-90 90 0"
         "ITEM_PROPERTY_CONNECTION_COUNT"      "0"
         "ITEM_PROPERTY_START_OPEN"      "1"
         "Facing"      "0 0 0"
      }
      "Item"
      {
         "Index"      "2"
         "Type"      "ITEM_OBSERVATION_ROOM"
         "Deletable"      "0"
         "VoxelPos"      "7 4 5"
         "LocalPos"      "0 0 0"
         "Angles"      "-90 0 0"
         "Facing"      "0 0 0"
      }
      [...]more items, user placed[...]
   }
   "Connections"
   {
      "Connection"
      {
         "Sender"      "<item index>"
         "Receiver"      "<item index>"
         "Type"      "<connection type>"
      }
      [...]more connections, user placed[...]
   }
}

**Things to do for fun**

1. Change your entire map to non-portable or portable by opening the .p2c in Notepad++ and using Ctrl+H (replace all) replacing the Portal0, Portal1, and Portal2 values all with 0 or 1 respectively.

2. Edit your puzzle creator made map in Hammer by pressing "Build" and then renaming the preview.vmf file it generates and opening that in Hammer. Be warned, it creates a ton of 128x128x128 blocks!

3. Check out the .3ds files in steamapps/common/portal2/portal2_dlc2/models/puzzlemaker

4. Make maps!

-A Puzzle Creator map open in Hammer showing all the blocks-

TODO: MOAR

Edits

5/10/12 - added info from Robdon and Djinndrache

Dang bro, you went super in depth on this, nice job homie.

Edit: Also forgot to mention me and happy an sleeper will mostly be posting anything we find in this thread about In-game Editor to hammer issues/cool things. Helps us out guys if you find anything as well. If you guys have any tutorial videos of these things, post them and we'll try and edit them into the op.

Please don't flood the thread though with a bunch of different videos that cover the same thing (aka check the op), unless of course your video does a better job.

Mr. Happy wrote:
PreviewDirty : HMMMM, dirty test chambers? Setting this to 1 didn't seem to do anything.

This variable is used set to true if the editor needs to re-create the preview thumbnail of the map. It's only used by the editor internally - you won't notice much of it

Amazing thread. Sticky.

Seems as if the devtest.p2c file is the basic building block of the pti base map, but contains info for "available items" to use in your map. Gonna play around with it and see if I can add more content to the menu.. or perhaps just change it. ex: eliminate gels for a map I don't want to use them for and add doors or some other content in their place in the editor menu.. Will let you know what I come up with.

Nice thread

Here are the switches used on the compile progs:

vbsp.exe: -entity_limit 1750
vvis.exe:
vrad.exe: -final -staticproppolys -staticproplighting -textureshadows -hdr

awesome Robdon! How did you find them, more importantly can you change them?
I'd love to add -low to all of them

I was wondering if anyone has figured out how to add a custom sp_lightboard_icons file to the map. I have tried adding it to the bsp like before but that does not seem to work.

Also Does anyone know how to add custom text when the map opens. Before I made a custom sp_tansition_list and changed the map's title and subtitle text.

@Patches34: game_text should d the trick?
I think that the lightboards icons are generated automatically with the new instance, but I have no idea if that's the case or not. And I cannot try it either.

lpfreaky90 wrote:
@Patches34: game_text should d the trick?
I think that the lightboards icons are generated automatically with the new instance, but I have no idea if that's the case or not. And I cannot try it either.

It's not the instance, it's the new client.dll that's modifying the lightboards.
So people don't worry about lightboards if you're uploading to the workshop.

I have several tutorials that I will post later today. (Hopefully)

So, if I understand right, we can create the logical part in a easy way in the PeTI, and later modify their aspect with hammer?
If yes.... can somebody give me a link to use Hammer in Portal 2? =D

Gugorrak wrote:
So, if I understand right, we can create the logical part in a easy way in the PeTI, and later modify their aspect with hammer?
If yes.... can somebody give me a link to use Hammer in Portal 2? =D

https://developer.valvesoftware.com/wiki/Level_Design_Introduction_(Portal_2)/Getting_Started

lpfreaky90 wrote:
Gugorrak wrote:

So, if I understand right, we can create the logical part in a easy way in the PeTI, and later modify their aspect with hammer?
If yes.... can somebody give me a link to use Hammer in Portal 2? =D

https://developer.valvesoftware.com/wiki/Level_Design_Introduction_(Portal_2)/Getting_Started

Thank you very much Brony partner =D

lpfreaky90 wrote:
awesome Robdon! How did you find them, more importantly can you change them?
I'd love to add -low to all of them

Hi,

I just use a process tracer to see whats going on.

Its 'Process Monitor' from here: http://technet.microsoft.com/en-gb/sysinternals

I just make a filter to only include vrad.exe etc process, and run a build, and then just right click a line and click 'properties'. It shows the command line used to start the process.

Robdon wrote:
Process Monitor

Another Sysinternals user!

Looks like it's not that hard to change the puzzle maker to include custom testing elements!
Check out this file:
steamapps\common\portal 2\portal2_dlc2\scripts\editoritems.txt

It's a keyvalues-formatted file that defines the behaviour of all of the stuff you can add in the editor. Model name, positioning rules, location on the parts list, names of instances to insert into the VMF, it's all there.

The only thing you can't do (as far as I can tell) is add more rows or columns to the parts list. But that's no huge deal, as you can free up 10 slots by removing redundant entries for the different buttons, cubes, fizzlers and gels. (And some things wouldn't even need their own slot. LP's death-fizzler, for example, could just be an extra option on the normal fizzler.)

I've only just started messing with this file so I don't fully understand it yet, but it looks very promising!

I've never used Hammer before and I have no idea how to use it, but after reading this...

It'd be possible to make maps using the editor, then open them up in Hammer to add extra things?

I mostly want to add checkpoints to some maps and I'd also like to choose which music plays. How hard would that be?

HMW wrote:
Looks like it's not that hard to change the puzzle maker to include custom testing elements!
Check out this file:
steamapps\common\portal 2\portal2_dlc2\scripts\editoritems.txt

It's a keyvalues-formatted file that defines the behaviour of all of the stuff you can add in the editor. Model name, positioning rules, location on the parts list, names of instances to insert into the VMF, it's all there.

The only thing you can't do (as far as I can tell) is add more rows or columns to the parts list. But that's no huge deal, as you can free up 10 slots by removing redundant entries for the different buttons, cubes, fizzlers and gels. (And some things wouldn't even need their own slot. LP's death-fizzler, for example, could just be an extra option on the normal fizzler.)

I've only just started messing with this file so I don't fully understand it yet, but it looks very promising!

Yes, but all the elements are hard coded into the client's base, so I don't know how you'll be able to edit the panel without some form of adjustment to the code.

Hang on a second folks... the maps are still built in VBSP, VVIS and VRAD, right? Well how does it compile them on Mac systems...? Have they ported the compilers or just been lazy and used some kind of WINE wrapper?