IDA: What's new in 5.7

Highlights

• IDAPython

  IDAPython has been modified a lot. We manually wrapped many new functions and classes. We documented all manually wrapped functions. Online documentation is here: idapython_docs

  Some "idaapi.cvar" variables have been moved to "idaapi.". In particular, the idaapi.cvar.cmd is now accessible as idaapi.cmd.

  Recent script box (Alt+7) and run python script (Alt+9) are replaced with Alt+F7 and Alt+F9 respectively

• Scripted plugins

  Plugins can be implemented in Python or IDC.

  http://hexblog.com/2010/03/scriptable_plugins.html
  
  See samples in "sdk\plugins\script_plg"

• Scripted processor modules

  Processor modules too can be implemented in Python or IDC. See:

  • sdk\module\script\ebc.py: EFI Byte code processor module. It works with EBC PE files.
  • sdk\module\script\msp430.py: MSP430 is a simple 27-instructions 16-bit RISC processor from TI.
  • sdk\module\script\proctemplate.py: Processor module script template

  We have addded many SDK functions to IDAPython to support processor modules.

• ARM module/Mach-O file format

  If you analyze iPhone/iPad file, the following improvements are essential:

  • parsing of the LC_DYLD_INFO and LC_ROUTINES loader commands (more names are recovered)
  • improved tracing of register values for ARM that discovers more references
  • improved offset auto-conversion: now you should see less false positives (e.g. no xrefs into middle of instructions)
  • indirect calls via register are resolved when possible, and a comment is added with the final address
  • better Thumb/ARM mode switch tracing
  • when splitting segments, T register is not reset to 0 but the old value at the split point is reused
  • support for PIC code generated by GCC 4.4.x (GOT loading)
  • various other fixes and improvements
  • ARM architecture options for disassembly can be configured in the UI (Processor-specific options), in IDA.CFG or on command line (see documentation; ARMv6 is default)
  • for Mach-O files and ELF files with EABI attributes the architecture is set automatically according to the flags in the binary
  • support for NEON SIMD instructions (available if ARMv7 is selected); they are commonly found in the current iPhone/iPad files
  • when ARMv7 is selected, a unified syntax is used for VFP and NEON instructions (Vxxx instead of old Fxxx)

• Custom data

  You can define your own data types. For more details, see:

  http://hexblog.com/2010/02/custom_data_types_and_formats_1.html

  Also check out "sdk\plugins\custview\custview.cpp" and IDAPython\examples\ex_custdata.py

• Bochs Linux debugger plugin

  We have ported the bochs debugger plugin to Linux. See this tutorial to learn how to configure and use it. The plugin functions similarly to the Windows bochs debugger plugin except that you need to set up environment variables and copy some Windows DLLs to Linux.

• PDB plugin

  The PDB plugin now works without having MS DIA DLLs registered in the system (i.e. no need to install the full Visual Studio). It is enough to either install VS 2005 or 2008 runtime redistributable or copy msdiaNN.dll into IDA's directory. It now imports much more types from PDB files with private symbols.

• WinDbg debugger plugin

  We improved WinDbg support:

  • Added non-invasive debugging support. You may find this feature useful if you want to attach to a program that is already being debugged.
  • The plugin will not automatically set the PDB path to %TEMP%\ida, one has to manually specify the symbol path (_NT_SYMBOL_PATH env var or .sympath command)
  • Minor bug fixes and speed improvements
  • Typing ".reload /f" will update the symbols state in the UI as well

• Recent scripts

  Alt-F7: Open script file
  Alt-F9: Opens the recent scripts chooser.
  

  • Ctrl+E to edit
  • Del to delete
  • Ins to insert a new script to the list

• New command line switches

  Please find more information here about the -t and -S switches
  

• Output window

  We replaced the listbox with a richtext control, allowing the users to:

  • double click on any identifier or address to jump to it if applicable
  • select and copy portions of text
  • search for text in the log

• Debugger

  • Jumping to debug names automatically creates code
  • Right click on a module in module window offers to jump to module base
  • debugger: added support for MSR, XMM and MMX registers
  • debugger: added support for model specific registers registers (win32)
  • breakpoints: breakpoints have a flag to allow the kernel to invalidate memory layout and contents before evaluating a script condition
  • added DbgByte/DbgWord/DbgDWord functions to read debuggee memory directly
  • the breakpoint list window also shows the instruction comment

Changelist

Processor Modules

• 6808: added new instructions from the 68HCS08 (aka 9S08) family
• ARM: added a switch pattern often found in EPOC files
• ARM: improved analysis of constant pool items - reduced false positives when converting data to offsets
• ARM: improved detection of ADRL macros
• ARM: handle some of the new idioms produced by GCC 4.4.0
• ARM: added support for NEON (aka Advanced SIMD) instructions, new to ARMv7 architecture
• ARM: architecture version can now be set; it is configured automatically for Mach-O files and ELF files with EABI attributes.
• ARM: many other improvements, switch patterns, etc
• EBC: EFI Byte Code processor module (written in Python)
• H8: 24-bit address operands were truncated to 16 bits
• MIPS: added MIPS16e instructions (jrc, jalrc, save, restore etc)
• MIPS: added ssnop instruction
• MIPS: more common instruction sequences are simplified
• MIPS: set proper types (float/double) for operands of FPU instructions - floating-point constants are now recognized and converted
• MIPS: substantially improved tracking of register values which recovers most of data and code cross-references in typical MIPS binaries
• MIPS: table-based switches are recognized and labeled
• MIPS: trace transitions between mips16 and mips32 code for better analysis
• MSP430: new processor module (written in Python)
• PC: improved detection of Delphi exception handlers
• PC: improved parsing of Visual C++ SEH handlers
• PC: recognize aligned stack prolog produced by GCC 4.x
• PIC: improved handling of FSRs for the PIC18xx series; added more chip configurations
• PPC: added G2 core (603e) instructions tlbld and tlbli
• PPC: more jump tables are recognized
• SuperH: added support for SH-2A architecture
• TMS320c54: added support for memory mappings (thanks to Sylvain Munaut)

File Formats

• .NET: made output more compatible with MSIL assembler
• ELF: added support for TLS relocations in x86_64 files
• ELF: handle MIPS files with mips16 functions
• ELF: handle more ARM relocations
• ELF: ARM: if EABI attributes are present (.ARM.attributes section), they're used to set up the ARM architecture options for disassembly
• Mach-O: rebasing a file (e.g. during debugging) could make some pointers invalid
• Mach-O: added support for X86_64_RELOC_SIGNED_n relocations
• Mach-O: added support for ARM BR24 and Thumb BR22 relocations
• Mach-O: cpusubtype field from the header is used to pre-set the ARM architecture version for disassembly
• PDB: vtable structures are created and added to class structures if that info is present in the PDB
• PDB: msdiaNN.dll can now be loaded without being registered, if found in PATH or default VC CRT install path
• PE: load configuration directory (which includes SEH information) is parsed and commented
• PE: recognize and load Phar Lap TNT DOS-Extender's 'PL' executables
• PE: IDA now allows to load corrupted files after a warning instead of aborting
• XCOFF: added support for weak extern symbols

Kernel

• added support for scripted processor modules
• added support for scripted plugins
• added support for 64-bit offsets with unknown base (they are mainly used in structure fields)
• added support for appcall with timeouts
• added support for custom data types and formats
• added support for locking of area pointers returned by the kernel
• improved the speed of handling idbs with huge number of segments
• improved argument propagation algorithm to handle indirect calls, including function pointers stored in stack variables
• jumping to exported entries will create functions automatically
• kernel: single-character strings were converted to Unicode too aggressively
• New command line switch '-t' to run IDA without an input file
• The "-S" switch now works with any supported script type. Users can pass arguments to scripts and access them via the "ARGV" global variable
• updated noret.cfg with several no-return functions used in Visual Basic programs (vbaErrorOverflow, vbaGenerateBoundsError etc.)
• the "Generate IDC" command retains function prototypes
• added FLIRT signatures for Visual C++ 2010 and C++Builder 2010

Scripts & SDK

• IDC: IDC definitions are not destroyed upon closing the database
• IDC: Added DbgByte(), DbgWord(), DbgWord() and DbgQword() to read program bytes from the debugger memory
• IDC: Added GetManyBytes() to read more than a byte from the database or the debugger memory
• SDK: added find_extlang_by_ext()
• SDK: added idp_notify::set_proc_options notification for more fine-grained configuration of processor modules
• SDK: added get_func_by_frame()
• SDK: added get_varcall_regs() for processors that use registers for vararg calls (like printf)
• SDK: added init_process/term_process/get_process_exit_code system independent functions
• SDK: added netnode::supdel_range() to delete range of supval elements
• SDK: added qfindfirst/qfindnext/qfindclose functions to enumerate files in a system independent way
• SDK: added qrename() to rename files. qrename() does not fail if the new file exists (unix behavior)
• SDK: added call_method to extlang
• SDK: added set_idc_func_ex(); this function makes it possible to register IDC functions that can work without an open idb; set_idc_func() is obsolete now and should not be used
• SDK: added support for complex offsets with subtraction from the base value (REFINFO_SUBTRACT)
• SDK: added VarGetClassName() to retrieve the class name of an IDC object
• SDK: class areaset_t can now be used by plugins and modules
• SDK: find_func_bounds() supports flag FIND_FUNC_IGNOREFN
• SDK: MIPS processor module now uses the same instruction numbers for 32-bit and 16-bit instructions. Plugins that rely on those numbers might have to be recompiled.
• SDK: processor modules that don't have instruction comments in the ida.int database will receive a get_autocmt notification.
• SDK: renamed all *_const functions to *_enum_member (add_const() is reserved in Visual C++ 2010)
• SDK: ua_stkvarN and add_stkvarN could fail when adding a stack variable with an odd offset

User Interface

• UI: added "follow pointer" context menu command in hex view
• UI: added a "jump to cross reference from" command (default shortcut Ctrl-J), useful when navigating large switch statements
• UI: it is now possible to generate flow and xref graphs in DOT format (see ida.cfg)
• UI: it is possible to add/delete breakpoints from the module names window
• UI: "Set callee" plugin (hotkey Alt-F11) now also works for ARM and MIPS processors
• UI: user can now enter type declaration for instruction operands
• UI: script boxes (Shift-F2, Alt-8) and other multi-line input boxes in IDA now auto-indent new lines
• UI: Output window is a ritchtext control. Users can double-click on identifiers / address to jump
• UI: Output window is now searchable (hotkey Alt-T and Ctrl-T)
• UI: The environment variable TVHEADLESS can be specified for the graphical version of IDA
• UI: Set function prototype (shortcut 'y') can also be used if cursor is positioned over a function name
• UI: Breakpoints window also shows the comment at the breakpoint's address
• UI: Replaced "File / IDC file" with "File / Script file" allowing the users to run any supported script file

Debugger

• Bochs debugger can be used under Linux and OS X
• Bochs debugger supports Bochs 2.4.5
• added "Jump to module base" to the modules list popup menu
• added "Analyze module" to the modules list popup menu
• added support for MMX/XMM registers (XMM regs only under windows)
• connection to a Windows CE device can be canceled
• win32 debugger: implemented accessing MSRs using the kernel debugger driver provided by Microsoft. Thanks to Alex Ionescu for the assistance!
• added support for reading/writing model specific registers (MSRs)
• WINDBG: changing the current thread using the "~Ns" command is now reflected in the UI
• WINDBG: Reloading symbols in the debugging engine using ".reload" will also update the debug names in IDA
• WINDBG: The plugin will propose to launch the dbgsrv.exe if debugging an x64 application
• WINDBG: Added support for non-invasive debugging

Bugfixes

• BUGFIX: AddHotKey() was broken under *nix
• BUGFIX: ARM: operand order for XScale MRA/MAR instructions was wrong
• BUGFIX: ARM: some instructions with PC-relative operand were decoded incorrectly in Thumb mode
• BUGFIX: ARM: some PC-relative load and store instructions were decoded incorrectly in Thumb mode
• BUGFIX: ARM: specifying a register pair in a function prototype would crash ida
• BUGFIX: ARM: Thumb-2 STRD instruction was sometimes decoded as LDRD
• BUGFIX: ELF: Solaris files with special values in sh_link field of section headers were not loaded correctly
• BUGFIX: nec850: jr was not stopping execution flow
• BUGFIX: PIC: 12F629 and 12F675 are 14-bit devices and so should be in pic14.cfg
• BUGFIX: Bochs plugin in disk image operation mode could not retrieve segment register base properly in Bochs 2.4.2
• BUGFIX: changing the storage type from sparse to regular could convert some zero-initialized bytes to uninitialized bytes
• BUGFIX: coff files with more than 0x8000 segments were loaded incorrectly (some symbols could not be resolved)
• BUGFIX: custom_viewer callbacks were getting wrong shift state information (should be 1-shift, 2-ctrl, 4-alt)
• BUGFIX: dbg_step_into and similar events were not generated if the operation was invoked interactively by the user (not from a plugin)
• BUGFIX: debug names were sent over the network repeatedly (remote debugging)
• BUGFIX: debugger: Linux debugger did not handle well programs which created a lot of short-lived threads
• BUGFIX: disabled menu items with shortcuts could appear in IDA after loading a new idb; this could lead to minor problems (like Esc not working in the decompiler)
• BUGFIX: disassembly lines for structure members that are arrays with some repeating values would be truncated
• BUGFIX: double clicking on a user-defined graph would lead to inconsistent idag state and create problems with focusing/unfocusing windows
• BUGFIX: TDS plugin: choosing CANCEL in instant debugging mode would crash IDA
• BUGFIX: Edit,Segment,Move current segment was doing nothing in some cases
• BUGFIX: editing a breakpoint could lead to a crash during remote debugging session
• BUGFIX: ELF (MIPS): in some cases HI16/LO16 relocation pairs were processed incorrectly
• BUGFIX: elf files were rebased incorrectly
• BUGFIX: even if the user requested octal represention of a number, IDA could still use decimal representation for values 8 and 9
• BUGFIX: fixed an integer overflow bug in the qnx file loader (thanks to Jason Geffner)
• BUGFIX: GDB debugger: ARM breakpoints were not working when connecting to OpenOCD
• BUGFIX: gdb plugin was crashing on empty reply from monitor command
• BUGFIX: get_prev_area() was broken
• BUGFIX: GUI: Floating forms could cause IDA GUI to crash if opened and closed many times
• BUGFIX: GUI: Hotkeys Alt-0 to Alt-9 (used to switch to a window) were not usuable even if no window was open and occupying that hotkey
• BUGFIX: IDA could crash trying to evaluate "eax++" (illegal postfix operation on a register)
• BUGFIX: IDA could crash while performing sp-analysis
• BUGFIX: IDA could crash with memory corruption
• BUGFIX: IDA could endlessly loop on corrupted databases
• BUGFIX: IDA could hang when instruction tracing is used with debuggers with the DBG_FLAG_DONT_DISTURB flag
• BUGFIX: IDA could hang when trying to display a type referring to itself
• BUGFIX: IDA could sometimes refuse to modify the stack pointer while the background analysis was on
• BUGFIX: IDA kernel was ignoring the "options" value set by the scriptable loaders in their accept_file()
• BUGFIX: IDA was not allowing names with dummy prefixes (like byte_...) for structure members; removed this limitation
• BUGFIX: IDA was setting 'use dup' checkbox while creating new arrays; now it remembers the last used value
• BUGFIX: IDA would try to continue to interact with the remote debugger server after network timeouts; now it immediately closes the connection because the protocol state is lost
• BUGFIX: idal could erronously complain about a corrupted configuration file
• BUGFIX: IDAPython: GetCharPrm(INF_PROCNAME) was broken
• BUGFIX: IDC DecodeInstruction() was not returning cmd.size attribute
• BUGFIX: IDC: SetHashLong() was broken
• BUGFIX: if a debugger session was ended with Alt-X, some debugger (and umimportant non-debugger) settings were not saved
• BUGFIX: if the network connection was dropped during a debugging session, IDA could crash in some cases
• BUGFIX: In rare cases, deleting the last segment could lead to a crash
• BUGFIX: in some cases automatic type propagation could overwrite the stack frame's return address field
• BUGFIX: it was impossible to connect to 64bit debugger server using 64bit version of IDA without a database
• BUGFIX: it was not possible to reload binary files
• BUGFIX: it was not possible to set an empty string as connection string for WinDbg debugger, if there was a saved default string.
• BUGFIX: it was possible to modify a readonly debugger register from a script
• BUGFIX: JAVA: IDA could not load some .class files with corrupted StackMapTable
• BUGFIX: MIPS: IDA was creating multi-instruction macros in delay slots, which was incorrect
• BUGFIX: MIPS: in some cases 16-bit negative immediate values were displayed as unsigned
• BUGFIX: MIPS: jalx was incorrectly stopping execution flow
• BUGFIX: normal functions were improperly detected as no-return for some processors with delayed slot instructions (e.g. SuperH)
• BUGFIX: only slightly damaged idb files could be repaired by IDA (normally IDA is able to repair badly damaged files too)
• BUGFIX: OSX: mac_server could interr in some cases when program exited unexpectedly
• BUGFIX: parse c header files: if a type name was used as a local structure member name in a C header file, it would be parsed incorrectly
• BUGFIX: parsing "typedef struct x x;" could create a circular dependency
• BUGFIX: PC: a wait instruction with prefixes was still lumped together with the next instruction
• BUGFIX: PC: assembler could not handle [reg-imm] while [reg+imm] was working ok
• BUGFIX: PIC: SFR definitions were not reloaded when opening a previous database
• BUGFIX: PowerPC: branch targets were truncated to 32 bits in 64-bit mode
• BUGFIX: pressing Cancel while initializing an instant debugger could hang ida
• BUGFIX: Python's GetReg and SetReg[Ex] did not work for non-x86 processors
• BUGFIX: retrieving objects with circular dependencies from the debugged program to IDC could lead to interr
• BUGFIX: SDK: unicode version of qstrlen() was broken
• BUGFIX: SDK: request_step_over/step() were not working properly with multithreaded applications
• BUGFIX: some SSE instructions were described incorrectly (missing CF_... flags)
• BUGFIX: SuperH: basic block boundaries were determined incorrectly, leading to wrong flow graphs
• BUGFIX: SuperH: return instructions were not defined correctly for big-endian variants
• BUGFIX: text version of IDA could crash at the exit time if all debugger plugins were deleted from the disk
• BUGFIX: the calculator could not properly display 64-bit and floating point results
• BUGFIX: the screen would not be always refreshed after modifying the very first byte of a segment using the right click menu
• BUGFIX: the selector list could not be displayed in the text version
• BUGFIX: TMS320C54 module could not display names located in segments with a non-zero segment base
• BUGFIX: TMS320C55x: some mov opcodes were decoded incorrectly
• BUGFIX: tracing systenter on Windows XP would fail
• BUGFIX: TXT: Open/Save file dialog could not list file name with wildcards in Windows
• BUGFIX: unmapping all mapped local types and saving the database would corrupt local type storage
• BUGFIX: wait_for_next_event() could hang if called without WFNE_SUSP and a suspending event occurred
• BUGFIX: win32: it was possible to attach to services and debug them only once; to attach the second time, IDA had to be restarted
• BUGFIX: windbg / kernel mode configuration was not being saved
• BUGFIX: Windbg plugin (with dump-files) and windmp loader were failing to read certain memory areas
• BUGFIX: IDC function xtol() function in IDC could not convert 64-bit values (in 64bit version of ida)