<- REVERSING

Alien Saboteaur

29 minutes to read

We are given a binary file called vm and a program called bin:

$ file *
bin: data
vm:  ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=10fb238b19d3a82b46536b51e47396525086a09c, for GNU/Linux 3.2.0, not stripped

Setup environment

The binary needs a recent version of Glibc in order to run:

$ ./vm
./vm: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./vm)

To solve this issue, we can run Ubuntu 22.04 in a Docker container and take the library and loader and patch the binary:

$ docker run -v "${PWD}:/opt" --rm -it ubuntu:22.04 bash
root@8ffbddd2ded9:/# /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libc.so.6
GNU C Library (Ubuntu GLIBC 2.35-0ubuntu3.1) stable release version 2.35.
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 11.2.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
For bug reporting instructions, please see:
<https://bugs.launchpad.net/ubuntu/+source/glibc/+bugs>.
root@8ffbddd2ded9:/# cp /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libc.so.6 /opt  
root@8ffbddd2ded9:/# exit
exit

$ patchelf --set-rpath . vm

$ patchelf --set-interpreter ld-linux-x86-64.so.2 vm

$ ./vm
Usage: ./chall file

Now we are able to run the provided program:

$ ./vm bin
[Main Vessel Terminal]  
< Enter keycode
>

Reverse engineering

Let’s open the binary in Ghidra. This is main:

int main(int argc, char **argv) {
  FILE *fp;
  size_t size;
  char *p_instructions;
  char *program;

  if (argc < 2) {
    printf("Usage: ./chall file");
                    /* WARNING: Subroutine does not return */  
    exit(1);
  }

  fp = fopen(argv[1], "rb");
  fseek(fp, 0, 2);
  size = ftell(fp);
  rewind(fp);
  p_instructions = (char *) malloc(size);
  fread(p_instructions, size, 1, fp);
  fclose(fp);
  program = (char *) vm_create(p_instructions, size);
  vm_run(program);

  return 0;
}

As can be seen, the program creates a virtual machine and runs the specified program (bin) there. This is vm_create:

undefined4 * vm_create(long p_instructions, long size) {
  undefined4 *vm;
  void *program;
  
  vm = (undefined4 *) malloc(0xa8);
  *vm = 0;
  *(undefined *) (vm + 1) = 0;
  vm[0x28] = 0;
  memset(vm + 2, 0, 0x80);
  program = calloc(0x10000, 1);
  *(void **) (vm + 0x24) = program;
  memcpy(*(void **) (vm + 0x24), (void *) (p_instructions + 3), size - 3);  
  program = calloc(0x200, 4);
  *(void **) (vm + 0x26) = program;

  return vm;
}

The variable names might be confusing. The important thing is that there are three chunks in memory (on the heap).

This is vm_run:

void vm_run(char *program) {
  while (program[4] == '\0') {  
    vm_step(program);
  }
}

It is calling vm_step:

void vm_step(char *param_1) {
  if (0x19 < *(byte *) ((ulong) *(uint *) param_1 + *(long *) (param_1 + 0x90))) {
    puts("dead");
                    /* WARNING: Subroutine does not return */
    exit(0);
  }

  (**(code **) (original_ops + (long) (int) (uint) *(byte *) ((ulong) *(uint *) param_1 + *(long *) (param_1 + 0x90)) * 8)(param_1);  
}

It is a bit unreadable. Basically, it is taking a byte from the input char array and using it as an offset to original_ops. Looking at the available functions, we can guess that the program executes one of these (the order is intended):

  1. vm_add
  2. vm_addi
  3. vm_sub
  4. vm_subi
  5. vm_mul
  6. vm_muli
  7. vm_div
  8. vm_cmp
  9. vm_cmp
  10. vm_inv
  11. vm_push
  12. vm_pop
  13. vm_mov
  14. vm_nop
  15. vm_exit
  16. vm_print
  17. vm_putc
  18. vm_je
  19. vm_jne
  20. vm_jle
  21. vm_jge
  22. vm_xor
  23. vm_store
  24. vm_load
  25. vm_input

Program parsing

Let’s take a look at one of these low-level functions:

void vm_addi(uint *param_1) {
  uint uVar1;
  byte bVar2;
  byte bVar3;

  bVar2 = u8((ulong) *param_1 + 2 + *(long *) (param_1 + 0x24));  
  uVar1 = param_1[(long) (int) (uint) bVar2 + 2];
  bVar2 = u8((ulong) *param_1 + 3 + *(long *) (param_1 + 0x24));
  bVar3 = u8((ulong) *param_1 + 1 + *(long *) (param_1 + 0x24));
  param_1[(long) (int) (uint) bVar3 + 2] = uVar1 + bVar2;
  *param_1 = *param_1 + 6;
}

For the moment, let’s focus on the fact that the param_1 pointer is increased in 6 (this is the program counter). So, we can guess that the instructions are 6-byte long. Moreover, in vm_create, the actual instructions start at offset 3:

  memcpy(*(void **) (vm + 0x24), (void *) (p_instructions + 3), size - 3);

Actually, if we read original_ops as undefined8[25], we will see the addresses of the vm_* functions:

                     original_ops                                    XREF[3]:     Entry Point(*),
                                                                                  vm_step:00102400(*),
                                                                                  vm_step:00102407(R)
00105020 64 1a 10        undefined8[25]
         00 00 00
         00 00 24
   00105020 64 1a 10 00 00  undefined8  0000000000101A64h     [0]           ?  ->  00101a64     XREF[3]:     Entry Point(*),
            00 00 00                                                                                         vm_step:00102400(*),  
                                                                                                             vm_step:00102407(R)
   00105028 24 1b 10 00 00  undefined8  0000000000101B24h     [1]           ?  ->  00101b24
            00 00 00
   00105030 d9 1b 10 00 00  undefined8  0000000000101BD9h     [2]           ?  ->  00101bd9
            00 00 00
   00105038 99 1c 10 00 00  undefined8  0000000000101C99h     [3]           ?  ->  00101c99
            00 00 00
   00105040 4e 1d 10 00 00  undefined8  0000000000101D4Eh     [4]           ?  ->  00101d4e
            00 00 00
   00105048 0f 1e 10 00 00  undefined8  0000000000101E0Fh     [5]           ?  ->  00101e0f
            00 00 00
   00105050 60 1f 10 00 00  undefined8  0000000000101F60h     [6]           ?  ->  00101f60
            00 00 00
   00105058 c5 1e 10 00 00  undefined8  0000000000101EC5h     [7]           ?  ->  00101ec5
            00 00 00
   00105060 7c 21 10 00 00  undefined8  000000000010217Ch     [8]           ?  ->  0010217c
            00 00 00
   00105068 25 20 10 00 00  undefined8  0000000000102025h     [9]           ?  ->  00102025
            00 00 00
   00105070 0c 22 10 00 00  undefined8  000000000010220Ch     [10]          ?  ->  0010220c
            00 00 00
   00105078 95 22 10 00 00  undefined8  0000000000102295h     [11]          ?  ->  00102295
            00 00 00
   00105080 27 23 10 00 00  undefined8  0000000000102327h     [12]          ?  ->  00102327
            00 00 00
   00105088 15 14 10 00 00  undefined8  0000000000101415h     [13]          ?  ->  00101415
            00 00 00
   00105090 e2 21 10 00 00  undefined8  00000000001021E2h     [14]          ?  ->  001021e2
            00 00 00
   00105098 fc 19 10 00 00  undefined8  00000000001019FCh     [15]          ?  ->  001019fc
            00 00 00
   001050a0 ae 19 10 00 00  undefined8  00000000001019AEh     [16]          ?  ->  001019ae
            00 00 00
   001050a8 92 16 10 00 00  undefined8  0000000000101692h     [17]          ?  ->  00101692
            00 00 00
   001050b0 59 17 10 00 00  undefined8  0000000000101759h     [18]          ?  ->  00101759
            00 00 00
   001050b8 20 18 10 00 00  undefined8  0000000000101820h     [19]          ?  ->  00101820
            00 00 00
   001050c0 e7 18 10 00 00  undefined8  00000000001018E7h     [20]          ?  ->  001018e7
            00 00 00
   001050c8 d2 15 10 00 00  undefined8  00000000001015D2h     [21]          ?  ->  001015d2
            00 00 00
   001050d0 93 14 10 00 00  undefined8  0000000000101493h     [22]          ?  ->  00101493
            00 00 00
   001050d8 31 15 10 00 00  undefined8  0000000000101531h     [23]          ?  ->  00101531
            00 00 00
   001050e0 33 14 10 00 00  undefined8  0000000000101433h     [24]          ?  ->  00101433
            00 00 00

The offsets are shown in the previous list.

Let’s take a look at the bin file:

$ xxd bin | head
00000000: 5577 5510 5b00 0000 0010 4d00 0000 0010  UwU.[.....M.....  
00000010: 6100 0000 0010 6900 0000 0010 6e00 0000  a.....i.....n...
00000020: 0010 2000 0000 0010 5600 0000 0010 6500  .. .....V.....e.
00000030: 0000 0010 7300 0000 0010 7300 0000 0010  ....s.....s.....
00000040: 6500 0000 0010 6c00 0000 0010 2000 0000  e.....l..... ...
00000050: 0010 5400 0000 0010 6500 0000 0010 7200  ..T.....e.....r.
00000060: 0000 0010 6d00 0000 0010 6900 0000 0010  ....m.....i.....
00000070: 6e00 0000 0010 6100 0000 0010 6c00 0000  n.....a.....l...
00000080: 0010 5d00 0000 0010 0a00 0000 0010 3c00  ..]...........<.
00000090: 0000 0010 2000 0000 0010 4500 0000 0010  .... .....E.....

The first three characters are skipped (UwU). Next, we have a 10 (16 in decimal format). This means that the program will execute vm_putc. Taking a look at the following characters, the program will print the banner ([Main Vessel Terminal]...) and the prompt character by character with vm_putc.

Let’s format the bin file a bit:

$ dd bs=1 skip=3 if=bin 2>/dev/null | xxd -g 6 -c 6  
00000000: 105b00000000  .[....
00000006: 104d00000000  .M....
0000000c: 106100000000  .a....
00000012: 106900000000  .i....
00000018: 106e00000000  .n....
0000001e: 102000000000  . ....
00000024: 105600000000  .V....
0000002a: 106500000000  .e....
00000030: 107300000000  .s....
00000036: 107300000000  .s....
0000003c: 106500000000  .e....
00000042: 106c00000000  .l....
00000048: 102000000000  . ....
0000004e: 105400000000  .T....
00000054: 106500000000  .e....
0000005a: 107200000000  .r....
00000060: 106d00000000  .m....
00000066: 106900000000  .i....
0000006c: 106e00000000  .n....
00000072: 106100000000  .a....
00000078: 106c00000000  .l....
0000007e: 105d00000000  .]....
00000084: 100a00000000  ......
0000008a: 103c00000000  .<....
00000090: 102000000000  . ....
00000096: 104500000000  .E....
0000009c: 106e00000000  .n....
000000a2: 107400000000  .t....
000000a8: 106500000000  .e....
000000ae: 107200000000  .r....
000000b4: 102000000000  . ....
000000ba: 106b00000000  .k....
000000c0: 106500000000  .e....
000000c6: 107900000000  .y....
000000cc: 106300000000  .c....
000000d2: 106f00000000  .o....
000000d8: 106400000000  .d....
000000de: 106500000000  .e....
000000e4: 102000000000  . ....
000000ea: 100a00000000  ......
000000f0: 103e00000000  .>....
000000f6: 102000000000  . ....
000000fc: 0c1ea00f0000  ......
00000102: 0c1c00000000  ......
00000108: 0c1d11000000  ......
0000010e: 181900000000  ......
00000114: 161e19000000  ......
0000011a: 011e1e010000  ......
00000120: 011c1c010000  ......
00000126: 131c1d2d0000  ...-..
0000012c: 0c1e04100000  ......
00000132: 0c1fa00f0000  ......
00000138: 0c1c00000000  ......
0000013e: 0c1d0a000000  ......
00000144: 0c1ba9000000  ......
0000014a: 0c1700000000  ......
00000150: 17191e000000  ......
00000156: 17181f000000  ......
0000015c: 1519191b0000  ......
00000162: 1119184e0000  ...N..
00000168: 105500000000  .U....
0000016e: 106e00000000  .n....
00000174: 106b00000000  .k....
0000017a: 106e00000000  .n....
00000180: 106f00000000  .o....
00000186: 107700000000  .w....
0000018c: 106e00000000  .n....
00000192: 102000000000  . ....
00000198: 106b00000000  .k....
0000019e: 106500000000  .e....
000001a4: 107900000000  .y....
000001aa: 106300000000  .c....
000001b0: 106f00000000  .o....
000001b6: 106400000000  .d....
000001bc: 106500000000  .e....
000001c2: 102100000000  .!....
000001c8: 100a00000000  ......
000001ce: 0e0000000000  ......
000001d4: 011e1e010000  ......
000001da: 011f1f010000  ......
000001e0: 011c1c010000  ......
000001e6: 131c1d380000  ...8..
000001ec: 0c0f00000000  ......
000001f2: 0a0f00000000  ......
000001f8: 0a0f00000000  ......
000001fe: 0a0f00000000  ......
00000204: 096503000000  .e....
0000020a: 0c1000000000  ......
00000210: 111f106c0000  ...l..
00000216: 105400000000  .T....
0000021c: 106500000000  .e....
00000222: 107200000000  .r....
00000228: 106d00000000  .m....
0000022e: 106900000000  .i....
00000234: 106e00000000  .n....
0000023a: 106100000000  .a....
00000240: 106c00000000  .l....
00000246: 102000000000  . ....
0000024c: 106200000000  .b....
00000252: 106c00000000  .l....
00000258: 106f00000000  .o....
0000025e: 106300000000  .c....
00000264: 106b00000000  .k....
0000026a: 106500000000  .e....
00000270: 106400000000  .d....
00000276: 102100000000  .!....
0000027c: 100a00000000  ......
00000282: 0e0000000000  ......
00000288: 0c1e77000000  ..w...
0000028e: 051e1e060000  ......
00000294: 0c1c00000000  ......
0000029a: 0c1ddc050000  ......
000002a0: 0c1b45000000  ..E...
000002a6: 17191e000000  ......
000002ac: 1519191b0000  ......
000002b2: 161e19000000  ......
000002b8: 011e1e010000  ......
000002be: 011c1c010000  ......
000002c4: 131c1d710000  ...q..
000002ca: 557945454545  UyEEEE
000002d0: 556545454545  UeEEEE
000002d6: 550045454545  U.EEEE
000002dc: 552b45454545  U+EEEE
000002e2: 553145454545  U1EEEE
000002e8: 552045454545  U EEEE
...

Program analysis

So, after the vm_putc instructions we have this:

000000ba: 106b00000000  .k....  putc  'k'
000000c0: 106500000000  .e....  putc  'e'
000000c6: 107900000000  .y....  putc  'y'
000000cc: 106300000000  .c....  putc  'c'
000000d2: 106f00000000  .o....  putc  'o'
000000d8: 106400000000  .d....  putc  'd'
000000de: 106500000000  .e....  putc  'e'
000000e4: 102000000000  . ....  putc  ' '
000000ea: 100a00000000  ......  putc  '\n'
000000f0: 103e00000000  .>....  putc  '>'
000000f6: 102000000000  . ....  putc  ' '
000000fc: 0c1ea00f0000  ......  mov   0x1e 0x0fa0
00000102: 0c1c00000000  ......  mov   0x1c 0x0000
00000108: 0c1d11000000  ......  mov   0x1d 0x0011
0000010e: 181900000000  ......  input 0x19
00000114: 161e19000000  ......  store 0x1e 0x19
0000011a: 011e1e010000  ......  addi  0x1e 0x1e 0x01  
00000120: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000126: 131c1d2d0000  ...-..  jle   0x1c 0x1d 0x2d
0000012c: 0c1e04100000  ......  mov   0x1e 0x1004
00000132: 0c1fa00f0000  ......  mov   0x1f 0x0fa0
00000138: 0c1c00000000  ......  mov   0x1c 0x0000
0000013e: 0c1d0a000000  ......  mov   0x1d 0x000a
00000144: 0c1ba9000000  ......  mov   0x1b 0x90ba
0000014a: 0c1700000000  ......  mov   0x17 0x0000
00000150: 17191e000000  ......  load  0x19 0x1e
00000156: 17181f000000  ......  load  0x18 0x1f
0000015c: 1519191b0000  ......  xor   0x19 0x19 0x1b
00000162: 1119184e0000  ...N..  je    0x19 0x18 0x4e
00000168: 105500000000  .U....  putc  'U'
0000016e: 106e00000000  .n....  putc  'n'
00000174: 106b00000000  .k....  putc  'k'
0000017a: 106e00000000  .n....  putc  'n'
00000180: 106f00000000  .o....  putc  'o'
00000186: 107700000000  .w....  putc  'w'
0000018c: 106e00000000  .n....  putc  'n'
00000192: 102000000000  . ....  putc  ' '
00000198: 106b00000000  .k....  putc  'k'
0000019e: 106500000000  .e....  putc  'e'
000001a4: 107900000000  .y....  putc  'y'
000001aa: 106300000000  .c....  putc  'c'
000001b0: 106f00000000  .o....  putc  'o'
000001b6: 106400000000  .d....  putc  'd'
000001bc: 106500000000  .e....  putc  'e'
000001c2: 102100000000  .!....  putc  '!'
000001c8: 100a00000000  ......  putc  '\n'
000001ce: 0e0000000000  ......  exit

To analyze how some opcodes work, we can use GDB:

$ gdb -q vm
Reading symbols from vm...
(No debugging symbols found in vm)
gef➤  break vm_input
Breakpoint 1 at 0x1433
gef➤  run bin
Starting program: ./vm bin
[Main Vessel Terminal]
< Enter keycode

Breakpoint 1, 0x0000555555555433 in vm_input ()

At this point, we can search for the program instructions:

gef➤  grep UwU
[+] Searching 'UwU' in memory
[+] In '[heap]'(0x55555555c000-0x55555557d000), permission=rw-  
  0x55555555d490 - 0x55555555d493  →   "UwU[...]"
gef➤  x/30gx 0x55555555d490
0x55555555d490: 0x0000005b10557755      0x10000000004d1000
0x55555555d4a0: 0x0069100000000061      0x0000006e10000000
0x55555555d4b0: 0x1000000000201000      0x0065100000000056
0x55555555d4c0: 0x0000007310000000      0x1000000000731000
0x55555555d4d0: 0x006c100000000065      0x0000002010000000
0x55555555d4e0: 0x1000000000541000      0x0072100000000065
0x55555555d4f0: 0x0000006d10000000      0x1000000000691000
0x55555555d500: 0x006110000000006e      0x0000006c10000000
0x55555555d510: 0x10000000005d1000      0x003c10000000000a
0x55555555d520: 0x0000002010000000      0x1000000000451000
0x55555555d530: 0x007410000000006e      0x0000006510000000
0x55555555d540: 0x1000000000721000      0x006b100000000020
0x55555555d550: 0x0000006510000000      0x1000000000791000
0x55555555d560: 0x006f100000000063      0x0000006410000000
0x55555555d570: 0x1000000000651000      0x000a100000000020

Then we can search for some of the values from vm_mov instructions:

gef➤  grep 0x0fa0
[+] Searching '\xa0\x0f' in memory
[+] In '[heap]'(0x55555555c000-0x55555557d000), permission=rw-
  0x55555555c500 - 0x55555555c508  →   "\xa0\x0f[...]"
  0x55555555d591 - 0x55555555d599  →   "\xa0\x0f[...]"
  0x55555555d5c7 - 0x55555555d5cf  →   "\xa0\x0f[...]"
  0x55555556159e - 0x5555555615a6  →   "\xa0\x0f[...]"
  0x5555555615d4 - 0x5555555615dc  →   "\xa0\x0f[...]"
[+] In './libc.so.6'(0x7ffff7dbb000-0x7ffff7f50000), permission=r-x  
  0x7ffff7dbdcdf - 0x7ffff7dbdce7  →   "\xa0\x0f[...]"
  ...
gef➤  x/20gx 0x55555555c4b0
0x55555555c4b0: 0x0000000000000000      0x0000000000000000
0x55555555c4c0: 0x0000000000000000      0x0000000000000000
0x55555555c4d0: 0x0000000000000000      0x0000000000000000
0x55555555c4e0: 0x0000000000000000      0x0000000000000000
0x55555555c4f0: 0x0000000000000000      0x0000001100000000
0x55555555c500: 0x0000000000000fa0      0x0000000000000000
0x55555555c510: 0x00005555555614a0      0x000055555555c530
0x55555555c520: 0x0000000000000000      0x0000000000000811
0x55555555c530: 0x0000000000000000      0x0000000000000000
0x55555555c540: 0x0000000000000000      0x0000000000000000

After a lot of debugging, we can figure out that the above memory layout is used as registers (0x1f stores 0x0000, 0x1e stores 0x0fa0, 0x1e stores 0x0011, 0x1d stores 0x0000 and so on). Moreover, we have a pointer to the opcode instructions:

gef➤  x/20gx 0x00005555555614a0
0x5555555614a0: 0x4d10000000005b10      0x0000611000000000  
0x5555555614b0: 0x0000000069100000      0x2010000000006e10
0x5555555614c0: 0x0000561000000000      0x0000000065100000
0x5555555614d0: 0x7310000000007310      0x0000651000000000
0x5555555614e0: 0x000000006c100000      0x5410000000002010
0x5555555614f0: 0x0000651000000000      0x0000000072100000
0x555555561500: 0x6910000000006d10      0x00006e1000000000
0x555555561510: 0x0000000061100000      0x5d10000000006c10
0x555555561520: 0x00000a1000000000      0x000000003c100000
0x555555561530: 0x4510000000002010      0x00006e1000000000

The other pointer points right below the registers area (probably it is the stack, which is not used for the moment).

Let’s analyze this part of the program:

000000fc: 0c1ea00f0000  ......  mov   0x1e 0x0fa0
00000102: 0c1c00000000  ......  mov   0x1c 0x0000
00000108: 0c1d11000000  ......  mov   0x1d 0x0011
0000010e: 181900000000  ......  input 0x19
00000114: 161e19000000  ......  store 0x1e 0x19
0000011a: 011e1e010000  ......  addi  0x1e 0x1e 0x01  
00000120: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000126: 131c1d2d0000  ...-..  jle   0x1c 0x1d 0x2d

Basically, it moves the values to the registers (as shown previously). Then it uses vm_input, which reads a single character from stdin:

void vm_input(uint *param_1) {
  byte bVar1;
  uint uVar2;

  uVar2 = getchar();
  bVar1 = u8((ulong) *param_1 + 1 + *(long *) (param_1 + 0x24));  
  param_1[(long) (int) (uint) bVar1 + 2] = uVar2;
  *param_1 = *param_1 + 6;
}

Then, it uses vm_store to store the character in the address pointed to by 0x1e. After that, registers 0x1e and 0x1c are increased in 1 (register 0x1c is used as a counter).

Then it uses vm_jle to test if value at 0x1c is less than or equal to the value at 0x1d. If so, it jumps to 0x2d? Let’s analyze vm_jle:

void vm_jle(uint *vm) {
  uint uVar1;
  byte bVar2;
  ushort uVar3;
  
  bVar2 = u8((ulong) *vm + 1 + *(long *) (vm + 0x24));
  uVar1 = vm[(long) (int) (uint)bVar2 + 2];
  bVar2 = u8((ulong) *vm + 2 + *(long *) (vm + 0x24));

  if (vm[(long) (int) (uint) bVar2 + 2] < uVar1) {
    *vm = *vm + 6;
  } else {
    uVar3 = u16((ulong) *vm + 3 + *(long *) (vm + 0x24));  
    *vm = ((uint) uVar3 * 2 + (uint) uVar3) * 2;
  }
}

We can see that in the case the condition matches, the program returns to the third argument times 2 and times 3. Therefore, jle 0x1c 0x1d 0x2d will jump to 0x2d * 6 = 0x010e if the condition matches. In fact, it makes sense. Let’s update the assembly code:

000000fc: 0c1ea00f0000  ......  mov   0x1e 0x0fa0
00000102: 0c1c00000000  ......  mov   0x1c 0x0000
00000108: 0c1d11000000  ......  mov   0x1d 0x0011
0000010e: 181900000000  ......  input 0x19
00000114: 161e19000000  ......  store 0x1e 0x19
0000011a: 011e1e010000  ......  addi  0x1e 0x1e 0x01
00000120: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000126: 131c1d2d0000  ...-..  jle   0x1c 0x1d 0x010e

First stage

So, the program executes a loop to read a total of 0x11 (17 in decimal) characters from stdin.

Let’s set a breakpoint at vm_load and enter some data:

gef➤  del
gef➤  break vm_load
Breakpoint 2 at 0x555555555531
gef➤  run
Starting program: ./vm bin
[Main Vessel Terminal]
< Enter keycode
> AAAAAAAAAAAAAAAAA

Breakpoint 2, 0x0000555555555531 in vm_load ()

Let’s see where it is stored:

gef➤  grep AAAAAAAAAAAAAAAA
[+] Searching 'AAAAAAAAAAAAAAAA' in memory
[+] In '[heap]'(0x55555555c000-0x55555557d000), permission=rw-  
  0x555555562440 - 0x555555562453  →   "AAAAAAAAAAAAAAAAA\n"
  0x5555555714b0 - 0x5555555714c3  →   "AAAAAAAAAAAAAAAAA\n"
gef➤  x/20gx 0x555555562440
0x555555562440: 0x4141414141414141      0x4141414141414141
0x555555562450: 0x0000000000000a41      0x0000000000000000
0x555555562460: 0x0000000000000000      0x0000000000000000
0x555555562470: 0x0000000000000000      0x0000000000000000
0x555555562480: 0x0000000000000000      0x0000000000000000
0x555555562490: 0x0000000000000000      0x0000000000000000
0x5555555624a0: 0x9acd99ca00000000      0xdcc19cf6cd9adbf6
0x5555555624b0: 0x000000c7de99cddd      0x0000000000000000
0x5555555624c0: 0x0000000000000000      0x0000000000000000
0x5555555624d0: 0x0000000000000000      0x0000000000000000

Alright. Moreover, these are the registers:

gef➤  x/20gx 0x55555555c4b0
0x55555555c4b0: 0x0000000000000000      0x0000000000000000  
0x55555555c4c0: 0x0000000000000000      0x0000000000000000
0x55555555c4d0: 0x0000000000000000      0x0000000000000000
0x55555555c4e0: 0x0000000000000000      0x0000000a00000000
0x55555555c4f0: 0x000000a900000000      0x0000000a00000000
0x55555555c500: 0x00000fa000001004      0x0000000000000000
0x55555555c510: 0x00005555555614a0      0x000055555555c530
0x55555555c520: 0x0000000000000000      0x0000000000000811
0x55555555c530: 0x0000000000000000      0x0000000000000000
0x55555555c540: 0x0000000000000000      0x0000000000000000

Actually, the values at 0x1f and 0x1e are offsets to the two buffers (taking the program instructions address as reference):

gef➤  x/4gx 0x00005555555614a0 + 0x0fa0
0x555555562440: 0x4141414141414141      0x4141414141414141  
0x555555562450: 0x0000000000000a41      0x0000000000000000
gef➤  x/4gx 0x00005555555614a0 + 0x1004
0x5555555624a4: 0xcd9adbf69acd99ca      0xde99cddddcc19cf6
0x5555555624b4: 0x00000000000000c7      0x0000000000000000

So, after entering some input data, these instructions are executed:

0000012c: 0c1e04100000  ......  mov   0x1e 0x1004
00000132: 0c1fa00f0000  ......  mov   0x1f 0x0fa0
00000138: 0c1c00000000  ......  mov   0x1c 0x0000
0000013e: 0c1d0a000000  ......  mov   0x1d 0x000a
00000144: 0c1ba9000000  ......  mov   0x1b 0x00a9
0000014a: 0c1700000000  ......  mov   0x17 0x0000
00000150: 17191e000000  ......  load  0x19 0x1e
00000156: 17181f000000  ......  load  0x18 0x1f
0000015c: 1519191b0000  ......  xor   0x19 0x19 0x1b
00000162: 1119184e0000  ...N..  je    0x19 0x18 0x01d4  
00000168: 105500000000  .U....  putc  'U'
0000016e: 106e00000000  .n....  putc  'n'
00000174: 106b00000000  .k....  putc  'k'
0000017a: 106e00000000  .n....  putc  'n'
00000180: 106f00000000  .o....  putc  'o'
00000186: 107700000000  .w....  putc  'w'
0000018c: 106e00000000  .n....  putc  'n'
00000192: 102000000000  . ....  putc  ' '
00000198: 106b00000000  .k....  putc  'k'
0000019e: 106500000000  .e....  putc  'e'
000001a4: 107900000000  .y....  putc  'y'
000001aa: 106300000000  .c....  putc  'c'
000001b0: 106f00000000  .o....  putc  'o'
000001b6: 106400000000  .d....  putc  'd'
000001bc: 106500000000  .e....  putc  'e'
000001c2: 102100000000  .!....  putc  '!'
000001c8: 100a00000000  ......  putc  '\n'
000001ce: 0e0000000000  ......  exit
000001d4: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000001da: 011f1f010000  ......  addi  0x1f 0x1f 0x01
000001e0: 011c1c010000  ......  addi  0x1c 0x1c 0x01
000001e6: 131c1d380000  ...8..  jle   0x1c 0x1d 0x0150

Here we have another loop. It takes a character from a buffer (at the address stored in 0x1e) and loads it into register 0x19. Then it takes a character from the address at register 0x1f (our input) and stores it into register 0x18. Then it computes a XOR operation between registers 0x19 and 0x1b (value 0xa9) and stores the result at 0x19. If the result matches with the character at register 0x18 (our input character), then it jumps to the bottom of the code snippet (which increases counters and loops again). Otherwise, it prints an error message and exits.

Since it is using XOR and we know the key and the expected result, let’s see what is the expected input:

$ python3 -q
>>> from pwn import p64, p8, xor
>>> res = p64(0xcd9adbf69acd99ca) + p64(0xde99cddddcc19cf6) + p8(0xc7)  
>>> key = p8(0xa9)
>>> xor(res, key)
b'c0d3_r3d_5hutd0wn'

With this password, we pass to another stage of the program:

$ ./vm bin
[Main Vessel Terminal]  
< Enter keycode
> c0d3_r3d_5hutd0wn
< Enter secret phrase
>

Curiously, the program uses a counter until 0x0a, so we only need to enter a total of 11 correct characters:

$ ./vm bin
[Main Vessel Terminal]  
< Enter keycode
> c0d3_r3d_5haaaaaa
< Enter secret phrase
>

Second stage

The program continues with these instructions:

000001ec: 0c0f00000000  ......  mov   0x0f 0x0000
000001f2: 0a0f00000000  ......  push  0x0f
000001f8: 0a0f00000000  ......  push  0x0f
000001fe: 0a0f00000000  ......  push  0x0f
00000204: 096503000000  .e....  inv   0x65 0x03
0000020a: 0c1000000000  ......  mov   0x10 0x0000
00000210: 111f106c0000  ...l..  je    0x1f 0x10 0x0288  
00000216: 105400000000  .T....  putc  'T'
0000021c: 106500000000  .e....  putc  'e'
00000222: 107200000000  .r....  putc  'r'
00000228: 106d00000000  .m....  putc  'm'
0000022e: 106900000000  .i....  putc  'i'
00000234: 106e00000000  .n....  putc  'n'
0000023a: 106100000000  .a....  putc  'a'
00000240: 106c00000000  .l....  putc  'l'
00000246: 102000000000  . ....  putc  ' '
0000024c: 106200000000  .b....  putc  'b'
00000252: 106c00000000  .l....  putc  'l'
00000258: 106f00000000  .o....  putc  'o'
0000025e: 106300000000  .c....  putc  'c'
00000264: 106b00000000  .k....  putc  'k'
0000026a: 106500000000  .e....  putc  'e'
00000270: 106400000000  .d....  putc  'd'
00000276: 102100000000  .!....  putc  '!'
0000027c: 100a00000000  ......  putc  '\n'
00000282: 0e0000000000  ......  exit

It is interesting how the program is pushing NULL values onto the stack. Then it uses vm_inv:

void vm_inv(uint *vm) {
  byte a;
  byte b;
  uint uVar1;
  uint uVar2;
  uint uVar3;
  long lVar4;

  a = u8((ulong)*vm + 1 + *(long *)(vm + 0x24));
  b = u8((ulong)*vm + 2 + *(long *)(vm + 0x24));

  if (b == 0) {
    uVar1 = 0;
  } else {
    uVar1 = vm[0x28];
    vm[0x28] = uVar1 - 1;
    uVar1 = *(uint *) ((ulong) (uVar1 - 1) * 4 + *(long *) (vm + 0x26));
  }

  if (b < 2) {
    uVar2 = 0;
  } else {
    uVar2 = vm[0x28];
    vm[0x28] = uVar2 - 1;
    uVar2 = *(uint *) ((ulong) (uVar2 - 1) * 4 + *(long *) (vm + 0x26));
  }

  if (b < 3) {
    uVar3 = 0;
  } else {
    uVar3 = vm[0x28];
    vm[0x28] = uVar3 - 1;
    uVar3 = *(uint *) ((ulong) (uVar3 - 1) * 4 + *(long *) (vm + 0x26));
  }

  lVar4 = syscall((ulong) a, (ulong) uVar1, (ulong) uVar2, (ulong) uVar3);  
  vm[0x21] = (uint) lVar4;
  *vm = *vm + 6;
}

The function looks a bit difficult to understand. The key here is the syscall function. Basically, vm_inv will run a syscall instruction. In this case, it is running sys_ptrace because $rax = 0x65 (more information at x64.syscall.sh). This syscall instruction will tell if the current program is being debugged:

gef➤  del
gef➤  run
Starting program: ./vm bin
[Main Vessel Terminal]
< Enter keycode
> c0d3_r3d_5hutd0wn
Terminal blocked!
[Inferior 1 (process 187597) exited normally]  
gef➤

It will block us if we are in a debugger… Actually, the program just checks that the error code of the syscall instruction is 0 or not. So, let’s bypass this:

gef➤  disassemble vm_inv
Dump of assembler code for function vm_inv:
   0x0000000000002025 <+0>:     endbr64
   0x0000000000002029 <+4>:     push   rbp
   0x000000000000202a <+5>:     mov    rbp,rsp
   0x000000000000202d <+8>:     sub    rsp,0x30
   0x0000000000002031 <+12>:    mov    QWORD PTR [rbp-0x28],rdi  
   ...
   0x0000000000002151 <+300>:   mov    rdi,rax
   0x0000000000002154 <+303>:   mov    eax,0x0
   0x0000000000002159 <+308>:   call   0x11d0 <syscall@plt>
   0x000000000000215e <+313>:   mov    edx,eax
   0x0000000000002160 <+315>:   mov    rax,QWORD PTR [rbp-0x28]
   0x0000000000002164 <+319>:   mov    DWORD PTR [rax+0x84],edx
   0x000000000000216a <+325>:   mov    rax,QWORD PTR [rbp-0x28]
   0x000000000000216e <+329>:   mov    eax,DWORD PTR [rax]
   0x0000000000002170 <+331>:   lea    edx,[rax+0x6]
   0x0000000000002173 <+334>:   mov    rax,QWORD PTR [rbp-0x28]
   0x0000000000002177 <+338>:   mov    DWORD PTR [rax],edx
   0x0000000000002179 <+340>:   nop
   0x000000000000217a <+341>:   leave
   0x000000000000217b <+342>:   ret
End of assembler dump.
gef➤  break *vm_inv+313
Breakpoint 1 at 0x215e
gef➤  run bin
Starting program: ./vm bin
[Main Vessel Terminal]
< Enter keycode
> c0d3_r3d_5hutd0wn

Breakpoint 1, 0x000055555555615e in vm_inv ()

This is the value of $rax (-1):

gef➤  p/x $rax
$1 = 0xffffffffffffffff

But inside the debugger we got the power:

gef➤  set $rax = 0
gef➤  continue
Continuing.
< Enter secret phrase  
>

As easy as that! This is the code after the sys_ptrace check:

00000288: 0c1e77000000  ..w...  mov   0x1e 0x0077
0000028e: 051e1e060000  ......  muli  0x1e 0x1e 0x06
00000294: 0c1c00000000  ......  mov   0x1c 0x0000
0000029a: 0c1ddc050000  ......  mov   0x1d 0x05dc
000002a0: 0c1b45000000  ..E...  mov   0x1b 0x0045
000002a6: 17191e000000  ......  load  0x19 0x1e
000002ac: 1519191b0000  ......  xor   0x19 0x19 0x1b
000002b2: 161e19000000  ......  store 0x1e 0x19
000002b8: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000002be: 011c1c010000  ......  addi  0x1c 0x1c 0x01
000002c4: 131c1d710000  ...q..  jle   0x1c 0x1d 0x02a6

With the above code, register 0x1e will be set to 0x02ca:

$ python3 -q
>>> hex(0x77 * 0x6)  
'0x2ca'

Then, 0x1c to 0x0000, 0x1d to 0x05dc and 0x1b to 0x45. After that, the program takes a byte at the address stored in register 0x1e and saves it at register 0x19, uses XOR with key at register 0x1b (0x45) and stores back the result from register 0x19 to the address in register 0x1e.

Notice that the target address refers to part of the program:

gef➤  x/30gx 0x55555555d490 + 0x2ca
0x55555555d75a: 0x4545457955000071      0x5545454545655545  
0x55555555d76a: 0x452b554545454500      0x4545453155454545
0x55555555d77a: 0x5545454545205545      0x4565554545454537
0x55555555d78a: 0x4545453655454545      0x5545454545205545
0x55555555d79a: 0x4537554545454526      0x4545452055454545
0x55555555d7aa: 0x5545454545315545      0x4535554545454565
0x55555555d7ba: 0x4545452d55454545      0x5545454545375545
0x55555555d7ca: 0x4536554545454524      0x4545452055454545
0x55555555d7da: 0x55454545454f5545      0x456555454545457b
0x55555555d7ea: 0x4554755b49454545      0x4945454545594945
0x55555555d7fa: 0x455c5d4545456158      0x45455c5b53454545
0x55555555d80a: 0x444545445b5b4445      0x5859564545445959
0x55555555d81a: 0x45454559494545d7      0x4945454566584945
0x55555555d82a: 0xd15a49454554755b      0x4545455f49454554
0x55555555d83a: 0x52454545665e4945      0x5a50524545455b51

Obviously, they are not valid opcode instructions:

000002ca: 557945454545  UyEEEE  
000002d0: 556545454545  UeEEEE
000002d6: 550045454545  U.EEEE
000002dc: 552b45454545  U+EEEE
000002e2: 553145454545  U1EEEE
000002e8: 552045454545  U EEEE
000002ee: 553745454545  U7EEEE
000002f4: 556545454545  UeEEEE
000002fa: 553645454545  U6EEEE
00000300: 552045454545  U EEEE
00000306: 552645454545  U&EEEE
0000030c: 553745454545  U7EEEE
...

But using XOR on these, the program decrypts a ciphertext that will transform into valid opcode instructions. Let’s do it in Python:

$ python3 -q
>>> from pwn import xor
>>>
>>> with open('bin', 'rb') as f:
...     program = f.read()
...
>>> program = program[3:]
>>>
>>> key = b'\x45'
>>> ct = program[0x2ca : 0x2ca + 0x5dc]
>>> instructions = xor(ct, key)
>>> instructions.hex()
'103c00000000102000000000104500000000106e00000000107400000000106500000000107200000000102000000000107300000000106500000000106300000000107200000000106500000000107400000000102000000000107000000000106800000000107200000000106100000000107300000000106500000000100a00000000103e000000001020000000000c1e301100000c1c000000000c1d24000000181900000000161e19000000011e1e010000011c1c010000131c1d9200000c1c000000000c1d230000000c1e301100000c1f941100000c1a000000000c1b2300000017141e00000017151f0000000a14000000000b13000000000c1230110000001212150000171112000000161e11000000161213000000011a1a010000011e1e010000011f1f010000131a1b9d00000c1e301100000c1ff81100000c1a000000000c1b2300000017141e0000000a1f000000000b0f00000000000f0f1c000017100f000000151414100000161e14000000011a1a010000011e1e010000131a1bae0000011c1c010000131c1d9900000c1e301100000c1f5c1200000c1a000000000c1b23000000170f1e00000017101f000000110f10c90000105700000000107200000000106f00000000106e00000000106700000000102100000000100a000000000e0000000000011a1a010000011e1e010000011f1f010000131a1bbe0000104100000000106300000000106300000000106500000000107300000000107300000000102000000000106700000000107200000000106100000000106e00000000107400000000106500000000106400000000102c00000000102000000000107300000000106800000000107500000000107400000000107400000000106900000000106e00000000106700000000102000000000106400000000106f00000000107700000000106e00000000102100000000100a000000000e0000000000454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545'  
>>> print('\n'.join(instructions[i : i + 6].hex() for i in range(0, len(instructions), 6)))
103c00000000
102000000000
104500000000
106e00000000
107400000000
106500000000
107200000000
102000000000
107300000000
106500000000
106300000000
107200000000
106500000000
107400000000
102000000000
107000000000
106800000000
107200000000
106100000000
107300000000
106500000000
100a00000000
103e00000000
102000000000
0c1e30110000
0c1c00000000
0c1d24000000
181900000000
161e19000000
011e1e010000
011c1c010000
131c1d920000
0c1c00000000
0c1d23000000
0c1e30110000
0c1f94110000
0c1a00000000
0c1b23000000
17141e000000
17151f000000
0a1400000000
0b1300000000
0c1230110000
001212150000
171112000000
161e11000000
161213000000
011a1a010000
011e1e010000
011f1f010000
131a1b9d0000
0c1e30110000
0c1ff8110000
0c1a00000000
0c1b23000000
17141e000000
0a1f00000000
0b0f00000000
000f0f1c0000
17100f000000
151414100000
161e14000000
011a1a010000
011e1e010000
131a1bae0000
011c1c010000
131c1d990000
0c1e30110000
0c1f5c120000
0c1a00000000
0c1b23000000
170f1e000000
17101f000000
110f10c90000
105700000000
107200000000
106f00000000
106e00000000
106700000000
102100000000
100a00000000
0e0000000000
011a1a010000
011e1e010000
011f1f010000
131a1bbe0000
104100000000
106300000000
106300000000
106500000000
107300000000
107300000000
102000000000
106700000000
107200000000
106100000000
106e00000000
107400000000
106500000000
106400000000
102c00000000
102000000000
107300000000
106800000000
107500000000
107400000000
107400000000
106900000000
106e00000000
106700000000
102000000000
106400000000
106f00000000
107700000000
106e00000000
102100000000
100a00000000
0e0000000000
454545454545
454545454545
454545454545
...

So, we have more code to analyze.

Third stage

These are the instructions:

000002ca: 103c00000000  .<....  putc  '<'
000002d0: 102000000000  . ....  putc  ' '
000002d6: 104500000000  .E....  putc  'E'
000002dc: 106e00000000  .n....  putc  'n'
000002e2: 107400000000  .t....  putc  't'
000002e8: 106500000000  .e....  putc  'e'
000002ee: 107200000000  .r....  putc  'r'
000002f4: 102000000000  . ....  putc  ' '
000002fa: 107300000000  .s....  putc  's'
00000300: 106500000000  .e....  putc  'e'
00000306: 106300000000  .c....  putc  'c'
0000030c: 107200000000  .r....  putc  'r'
00000312: 106500000000  .e....  putc  'e'
00000318: 107400000000  .t....  putc  't'
0000031e: 102000000000  . ....  putc  ' '
00000324: 107000000000  .p....  putc  'p'
0000032a: 106800000000  .h....  putc  'h'
00000330: 107200000000  .r....  putc  'r'
00000336: 106100000000  .a....  putc  'a'
0000033c: 107300000000  .s....  putc  's'
00000342: 106500000000  .e....  putc  'e'
00000348: 100a00000000  ......  putc  '\n'
0000034e: 103e00000000  .>....  putc  '>'
00000354: 102000000000  . ....  putc  ' '
0000035a: 0c1e30110000  ..0...  mov   0x1e 0x1130
00000360: 0c1c00000000  ......  mov   0x1c 0x0000
00000366: 0c1d24000000  ..$...  mov   0x1d 0x0024
0000036c: 181900000000  ......  input 0x19
00000372: 161e19000000  ......  store 0x1e 0x19
00000378: 011e1e010000  ......  addi  0x1e 0x1e 0x01
0000037e: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000384: 131c1d920000  ......  jle   0x1c 0x1d 0x36c  
0000038a: 0c1c00000000  ......  mov   0x1c 0x0000
00000390: 0c1d23000000  ..#...  mov   0x1d 0x0023
00000396: 0c1e30110000  ..0...  mov   0x1e 0x1130
0000039c: 0c1f94110000  ......  mov   0x1f 0x1194
000003a2: 0c1a00000000  ......  mov   0x1a 0x0000
000003a8: 0c1b23000000  ..#...  mov   0x1b 0x0023
000003ae: 17141e000000  ......  load  0x14 0x1e
000003b4: 17151f000000  ......  load  0x15 0x1f
000003ba: 0a1400000000  ......  push  0x14
000003c0: 0b1300000000  ......  pop   0x13
000003c6: 0c1230110000  ..0...  mov   0x12 0x1130
000003cc: 001212150000  ......  add   0x12 0x12 0x15
000003d2: 171112000000  ......  load  0x11 0x12
000003d8: 161e11000000  ......  store 0x1e 0x11
000003de: 161213000000  ......  store 0x12 0x13
000003e4: 011a1a010000  ......  addi  0x1a 0x1a 0x01
000003ea: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000003f0: 011f1f010000  ......  addi  0x1f 0x1f 0x01
000003f6: 131a1b9d0000  ......  jle   0x1a 0x1b 0x3ae
000003fc: 0c1e30110000  ..0...  mov   0x1e 0x1130
00000402: 0c1ff8110000  ......  mov   0x1f 0x11f8
00000408: 0c1a00000000  ......  mov   0x1a 0x0000
0000040e: 0c1b23000000  ..#...  mov   0x1b 0x0023
00000414: 17141e000000  ......  load  0x14 0x1e
0000041a: 0a1f00000000  ......  push  0x1f
00000420: 0b0f00000000  ......  pop   0x0f
00000426: 000f0f1c0000  ......  add   0x0f 0x0f 0x1c
0000042c: 17100f000000  ......  load  0x10 0x0f
00000432: 151414100000  ......  xor   0x14 0x14 0x10
00000438: 161e14000000  ......  store 0x1e 0x14
0000043e: 011a1a010000  ......  addi  0x1a 0x1a 0x01
00000444: 011e1e010000  ......  addi  0x1e 0x1e 0x01
0000044a: 131a1bae0000  ......  jle   0x1a 0x1b 0x414
00000450: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000456: 131c1d990000  ......  jle   0x1c 0x1d 0x396
0000045c: 0c1e30110000  ..0...  mov   0x1e 0x1130
00000462: 0c1f5c120000  ..\...  mov   0x1f 0x125c
00000468: 0c1a00000000  ......  mov   0x1a 0x0000
0000046e: 0c1b23000000  ..#...  mov   0x1b 0x0023
00000474: 170f1e000000  ......  load  0x0f 0x1e
0000047a: 17101f000000  ......  load  0x10 0x1f
00000480: 110f10c90000  ......  je    0x0f 0x10 0x4b6
00000486: 105700000000  .W....  putc  'W'
0000048c: 107200000000  .r....  putc  'r'
00000492: 106f00000000  .o....  putc  'o'
00000498: 106e00000000  .n....  putc  'n'
0000049e: 106700000000  .g....  putc  'g'
000004a4: 102100000000  .!....  putc  '!'
000004aa: 100a00000000  ......  putc  '\n'
000004b0: 0e0000000000  ......  exit
000004b6: 011a1a010000  ......  addi  0x1a 0x1a 0x01
000004bc: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000004c2: 011f1f010000  ......  addi  0x1f 0x1f 0x01
000004c8: 131a1bbe0000  ......  jle   0x1a 0x1b 0x474
000004ce: 104100000000  .A....  putc  'A'
000004d4: 106300000000  .c....  putc  'c'
000004da: 106300000000  .c....  putc  'c'
000004e0: 106500000000  .e....  putc  'e'
000004e6: 107300000000  .s....  putc  's'
000004ec: 107300000000  .s....  putc  's'
000004f2: 102000000000  . ....  putc  ' '
000004f8: 106700000000  .g....  putc  'g'
000004fe: 107200000000  .r....  putc  'r'
00000504: 106100000000  .a....  putc  'a'
0000050a: 106e00000000  .n....  putc  'n'
00000510: 107400000000  .t....  putc  't'
00000516: 106500000000  .e....  putc  'e'
0000051c: 106400000000  .d....  putc  'd'
00000522: 102c00000000  .,....  putc  ','
00000528: 102000000000  . ....  putc  ' '
0000052e: 107300000000  .s....  putc  's'
00000534: 106800000000  .h....  putc  'h'
0000053a: 107500000000  .u....  putc  'u'
00000540: 107400000000  .t....  putc  't'
00000546: 107400000000  .t....  putc  't'
0000054c: 106900000000  .i....  putc  'i'
00000552: 106e00000000  .n....  putc  'n'
00000558: 106700000000  .g....  putc  'g'
0000055e: 102000000000  . ....  putc  ' '
00000564: 106400000000  .d....  putc  'd'
0000056a: 106f00000000  .o....  putc  'o'
00000570: 107700000000  .w....  putc  'w'
00000576: 106e00000000  .n....  putc  'n'
0000057c: 102100000000  .!....  putc  '!'
00000582: 100a00000000  ......  putc  '\n'
00000588: 0e0000000000  ......  exit

This part should be clear:

0000035a: 0c1e30110000  ..0...  mov   0x1e 0x1130
00000360: 0c1c00000000  ......  mov   0x1c 0x0000
00000366: 0c1d24000000  ..$...  mov   0x1d 0x0024
0000036c: 181900000000  ......  input 0x19
00000372: 161e19000000  ......  store 0x1e 0x19
00000378: 011e1e010000  ......  addi  0x1e 0x1e 0x01
0000037e: 011c1c010000  ......  addi  0x1c 0x1c 0x01
00000384: 131c1d920000  ......  jle   0x1c 0x1d 0x36c

It is just taking user input again (size 0x24 bytes). The rest of the code is worth drawing it into a graph:

Alien Saboteaur 1

As can be seen, there is a loop on register 0x1c that contains two inner loops (on register 0x1a). Notice that register 0x1e contains the address where our input data is stored at (0x1130).

Python implementation

Let’s analyze the first inner loop. First of all, register 0x1f contains 0x1194, which represents this buffer:

gef➤  x/6gx 0x00005555555614a0 + 0x1194
0x555555562634: 0x0e1d00070a0f1913      0x14181f0b010c1016  
0x555555562644: 0x122204211a1c0908      0x1715020620111b05
0x555555562654: 0x0000000003231e0d      0x0000000000000000

This is the code for the first inner loop:

000003ae: 17141e000000  ......  load  0x14 0x1e
000003b4: 17151f000000  ......  load  0x15 0x1f
000003ba: 0a1400000000  ......  push  0x14
000003c0: 0b1300000000  ......  pop   0x13
000003c6: 0c1230110000  ..0...  mov   0x12 0x1130
000003cc: 001212150000  ......  add   0x12 0x12 0x15
000003d2: 171112000000  ......  load  0x11 0x12
000003d8: 161e11000000  ......  store 0x1e 0x11
000003de: 161213000000  ......  store 0x12 0x13
000003e4: 011a1a010000  ......  addi  0x1a 0x1a 0x01
000003ea: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000003f0: 011f1f010000  ......  addi  0x1f 0x1f 0x01
000003f6: 131a1b9d0000  ......  jle   0x1a 0x1b 0x3ae

Let’s break it into pieces:

  • Loads bytes from user input (0x1130) and stored buffer (0x1194) into registers 0x14 and 0x15, respectively:
000003ae: 17141e000000  ......  load  0x14 0x1e
000003b4: 17151f000000  ......  load  0x15 0x1f
  • Copies 0x14 to 0x13:
000003ba: 0a1400000000  ......  push  0x14  
000003c0: 0b1300000000  ......  pop   0x13
  • Uses register 0x15 as an offset to the user data (0x1130) and stores the position in 0x12:
000003c6: 0c1230110000  ..0...  mov   0x12 0x1130
000003cc: 001212150000  ......  add   0x12 0x12 0x15
  • Stores the character at address pointed to by 0x12 into 0x11 and stores this character in the address pointed to by 0x1e. Also, replaces the character at offset the value of register 0x12 with the original character (stored in 0x13):
000003d2: 171112000000  ......  load  0x11 0x12  
000003d8: 161e11000000  ......  store 0x1e 0x11
000003de: 161213000000  ......  store 0x12 0x13
  • Increases counters and checks the loop condition:
000003e4: 011a1a010000  ......  addi  0x1a 0x1a 0x01
000003ea: 011e1e010000  ......  addi  0x1e 0x1e 0x01
000003f0: 011f1f010000  ......  addi  0x1f 0x1f 0x01
000003f6: 131a1b9d0000  ......  jle   0x1a 0x1b 0x3ae

In breaf, it is a swap, so our input data will be shuffled according to a fix pattern. We can translate into Python code, which will be much more readable:

    for i in range(0x24):
        pos = orders[i]
        data[i], data[pos] = data[pos], data[i]

Let’s move on to the second loop. Now 0x1f points to another buffer (0x11f8):

gef➤  x/6gx 0x00005555555614a0 + 0x11f8
0x555555562698: 0xebdefb01b247b016      0x216e7c105d5b5d82  
0x5555555626a8: 0xd7d423362a45e75f      0xcb5ee7ed11a3d526
0x5555555626b8: 0x00000000e2dd9fdb      0x0000000000000000

This is the relevant code:

00000414: 17141e000000  ......  load  0x14 0x1e
0000041a: 0a1f00000000  ......  push  0x1f
00000420: 0b0f00000000  ......  pop   0x0f
00000426: 000f0f1c0000  ......  add   0x0f 0x0f 0x1c
0000042c: 17100f000000  ......  load  0x10 0x0f
00000432: 151414100000  ......  xor   0x14 0x14 0x10
00000438: 161e14000000  ......  store 0x1e 0x14
0000043e: 011a1a010000  ......  addi  0x1a 0x1a 0x01
00000444: 011e1e010000  ......  addi  0x1e 0x1e 0x01
0000044a: 131a1bae0000  ......  jle   0x1a 0x1b 0x414

This one should be easier to understand, but let’s break it into pieces again:

  • Loads a shuffled character:
00000414: 17141e000000  ......  load  0x14 0x1e
  • Copies the address of the second buffer from 0x1f to 0x0f:
0000041a: 0a1f00000000  ......  push  0x1f  
00000420: 0b0f00000000  ......  pop   0x0f
  • Increases the address stored at 0x0f in an offset equal to the value in 0x1c (the outer loop’s counter):
00000426: 000f0f1c0000  ......  add   0x0f 0x0f 0x1c
  • Loads the character at 0x0f into 0x10 and applies XOR to the value at 0x14. The result is stored at 0x14:
0000042c: 17100f000000  ......  load  0x10 0x0f
00000432: 151414100000  ......  xor   0x14 0x14 0x10
  • Stores the value at 0x14 in the address pointed to by 0x1e (user data):
00000438: 161e14000000  ......  store 0x1e 0x14
  • Increases counters and checks loop conditions:
0000043e: 011a1a010000  ......  addi  0x1a 0x1a 0x01
00000444: 011e1e010000  ......  addi  0x1e 0x1e 0x01
0000044a: 131a1bae0000  ......  jle   0x1a 0x1b 0x414

In Python, this loop can be written as follows:

    for i in range(0x24):
        data[i] ^= xors[k]

Notice that the inner loops are within another loop. So, summarizing, this is how tihs part looks like in Python:

orders = b''.join(map(p64, [0x0e1d00070a0f1913, 0x14181f0b010c1016,
                  0x122204211a1c0908, 0x1715020620111b05, 0x0000000003231e0d]))  

xors = b''.join(map(p64, [0xebdefb01b247b016, 0x216e7c105d5b5d82,
                0xd7d423362a45e75f, 0xcb5ee7ed11a3d526, 0x00000000e2dd9fdb]))

for k in range(0x24):
    for i in range(0x24):
        pos = orders[i]
        data[i], data[pos] = data[pos], data[i]

    for i in range(0x24):
        data[i] ^= xors[k]

After these loops, the program takes another buffer (0x125c):

gef➤  x/6gx 0x00005555555614a0 + 0x125c
0x5555555626fc: 0x355640334a775d31      0x32666e6e355d3775  
0x55555556270c: 0x79316a616670366c      0x3233606c7f70315d
0x55555556271c: 0x000000005d317b36      0x0000000000000000

It is just comparing the result of all the operations on the user buffer (0x1130) with the above one. If everything matches, the program will succeed; otherwise, the program will say Wrong!.

Just to complete the Python implementation, this is the code for the third stage:

#!/usr/bin/env python3

from pwn import p64


orders = b''.join(map(p64, [0x0e1d00070a0f1913, 0x14181f0b010c1016,
                  0x122204211a1c0908, 0x1715020620111b05, 0x0000000003231e0d]))

xors = b''.join(map(p64, [0xebdefb01b247b016, 0x216e7c105d5b5d82,
                0xd7d423362a45e75f, 0xcb5ee7ed11a3d526, 0x00000000e2dd9fdb]))

expected_result = b''.join(map(p64, [0x355640334a775d31, 0x32666e6e355d3775,
                           0x79316a616670366c, 0x3233606c7f70315d, 0x000000005d317b36]))  

data = list(input('< Enter secret phrase\n> ').encode())

for k in range(0x24):
    for i in range(0x24):
        pos = orders[i]
        data[i], data[pos] = data[pos], data[i]

    for i in range(0x24):
        data[i] ^= xors[k]

for i in range(0x24):
    if data[i] != expected_result[i]:
        print('Wrong!')
        exit()

print('Access granted, shutting down!')
exit()

Reversing the algorithm

Great, now we only need to reverse the algorithm. This is relatively easy, since all operations involved are reversible (XOR and swap), so we can take the expected output and find the expected input. We only need to “reverse” all operations:

#!/usr/bin/env python3

from pwn import p64


orders = b''.join(map(p64, [0x0e1d00070a0f1913, 0x14181f0b010c1016,
                  0x122204211a1c0908, 0x1715020620111b05, 0x0000000003231e0d]))

xors = b''.join(map(p64, [0xebdefb01b247b016, 0x216e7c105d5b5d82,
                0xd7d423362a45e75f, 0xcb5ee7ed11a3d526, 0x00000000e2dd9fdb]))

expected_result = b''.join(map(p64, [0x355640334a775d31, 0x32666e6e355d3775,
                           0x79316a616670366c, 0x3233606c7f70315d, 0x000000005d317b36]))  

data = list(expected_result)

for k in range(0x24 - 1, -1, -1):
    for i in range(0x24):
        data[i] ^= xors[k]

    for i in range(0x24 - 1, -1, -1):
        pos = orders[i]
        data[i], data[pos] = data[pos], data[i]

print(bytes(data))

Flag

If we run the above code, we will find the flag:

$ python3 solve.py
b'HTB{5w1rl_4r0und_7h3_4l13n_by73c0d3}\x00\x00\x00\x00'

Obviously, it also works in the Python implementation and in the original vm binary and bin program:

$ python3 test.py
< Enter secret phrase
> HTB{5w1rl_4r0und_7h3_4l13n_by73c0d3}  
Access granted, shutting down!

$ ./vm bin
[Main Vessel Terminal]
< Enter keycode
> c0d3_r3d_5hutd0wn
< Enter secret phrase
> HTB{5w1rl_4r0und_7h3_4l13n_by73c0d3}
Access granted, shutting down!