Malware Behaviour: Flag Hints

Advanced malware behavior analysis using dynamic reverse engineering techniques including process forking, network communication, library preloading, and binary unpacking.

Lab Overview

A CTF lab focusing on advanced malware behavior analysis. In your home directory you will find some binaries that you need to reverse engineer in order to determine the password that the program expects. Once you have found the password, run the program and enter the password to receive the file.

This lab covers advanced dynamic analysis techniques including process forking, network communication, library preloading, and binary unpacking. You will work with various malware behaviors and learn how to analyze them using GDB and other reverse engineering tools.

Authors: Thalita Vergilio, Tom Shaw, and Z. Cliffe Schreuders
License: CC BY-SA 4.0
Difficulty: intermediate
CyBOK Knowledge Areas: MAT: Malware Taxonomy dimensions kinds MAT: Malware Analysis analysis techniques analysis environments
Tags: malware-analysis dynamic-analysis process-forking network-analysis library-preloading binary-unpacking ctf

Contents

    Advanced Analysis Techniques

    Before attempting the CTF challenges, you’ll need to understand several advanced techniques used in malware analysis.

    GDB Fork Mode

    When analyzing programs that create child processes, you need to configure GDB to follow the child process:

    set follow-fork-mode child

    This tells GDB to debug the child process instead of the parent when a fork occurs.

    Library Preloading (LD_PRELOAD)

    LD_PRELOAD allows you to override system functions by loading your own shared library first. To create a shared library:

    gcc -shared -fPIC -o libname.so source.c

    To use it:

    LD_PRELOAD=./libname.so ./program

    Network Analysis

    For network-based challenges, you can use netcat to listen for connections:

    nc -l 8080

    This listens on port 8080 for incoming connections.

    CTF Challenges

    Tip: Here are some tips to help you find the flags:

    Ch12Covert_ForkFollow

    Hint: Remember to set the follow-fork mode to ‘child’ in GDB.

    Hint: Put a break on the cmp that decides whether to print the password or not.

    Hint: When it stops, check what is being compared.

    Hint: Watch the size of the data you are examining (this is randomly assigned, but it could be a word, a double word, etc).

    Ch12Covert_ForkPipe

    Hint: You need to set the follow-fork mode to ‘child’ again.

    Hint: You also need to enter a really long password (you will see why when you start debugging the program).

    Hint: Examine the try_command() function.

    Hint: Break at the line that compares dl and al.

    Hint: Now you can either work with these and the ‘set’ command, or look further up in the code for values of interest.

    Ch11MalBeh_NetcatShovel

    Hint: This one is easy. Open a new tab and run a netcat command to listen on port 8080.

    Hint: Run the challenge.

    Hint: Check the other tab for the password.

    Ch18PackUnp_UnpackEasy

    Hint: Copy the file to the user’s home directory to remove the setuid.

    Hint: Use UPX to unpack it.

    Hint: Run GDB at that location.

    Hint: Find the function that compares the string entered to the password. Note that there is no function name, only a memory address, but you can guess by the arguments to the function and the instructions afterwards that it is probably strcmp().

    Hint: You know what to do next 🙂

    Hint: Remember to run the program again from the challenges directory to get the password.

    Ch11MalBeh_LdPreloadGetUID

    Hint: Watch the LD_PRELOAD Demo lecture first!

    Hint: Copy the challenge executable to your home directory.

    Hint: In your home directory, create a file that implements getuid().

    Hint: Compile as a 32-bit dynamic library.

    Hint: If you try to run ldd, it will probably fail saying your dynamic library has the wrong ELF class. Ignore that.

    Hint: Run the challenge program from the home directory using your preloaded library. The password will be printed on the screen. Run it again from the challenges directory and enter the correct password.

    Ch11MalBeh_LdPreloadRand

    Hint: Follow the same procedure as the previous one.