Thank you. That helps some, but unfortunately, it isn't enough.
I think the following code works for adding it to the registry:
!define REG_GE "SOFTWARE\Microsoft\Windows\CurrentVersion\GameUX\Games"
WriteRegStr HKLM "${REG_GE}\${GAME_GUID}" "AppExePath" "$INSTDIR\${GAME_EXE}"
WriteRegStr HKLM "${REG_GE}\${GAME_GUID}" "ApplicationId" "${GAME_GUID}"
WriteRegStr HKLM "${REG_GE}\${GAME_GUID}" "ConfigApplicationPath" "$INSTDIR"
WriteRegStr HKLM "${REG_GE}\${GAME_GUID}" "ConfigInstallType" "4"
WriteRegStr HKLM "${REG_GE}\${GAME_GUID}" "Title" "${GAME_TITLE}"
However, I don't see how to add tasks to the game explorer for all users after that. Once the above registry keys have been added, the game appears in the Game Explorer, but it shows a default icon for it, and it creates a directory for the current user to put the play link into (%USERPROFILE%\AppData\Roaming\Microsoft\Windows\GameExplorer\
GAME_GUID\PlayTasks\0), but even if I put a link into a similar directory under the "all users" AppData (I think "C:\ProgramData" is the Vista default for that), it doesn't pick it up.