Jump to content
Nytro

kbd-audio

Recommended Posts

kbd-audio Build Status

Description

This is a collection of command-line and GUI tools for capturing and analyzing audio data.

Keytap

The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone.

Check this blog post for more details:

Keytap: description and some random thoughts

Video: short demo of Keytap in action

Keytap2

The keytap2 tool is another interesting tool for recovering text from audio. It does not require training data - instead it uses statistical information about the frequencies of the letters and n-grams in the English language. The tool is still in development, but you can see a short demonstration here:

Video: Keytap2 - recovering text from typing sound (7:50)

CTF: can you guess the text being typed?

Build instructions

Dependencies:

  • SDL2 - used to capture audio and to open GUI windows libsdl

    [Ubuntu]
    $ sudo apt install libsdl2-dev
    
    [Mac OS with brew]
    $ brew install sdl2  
    
  • FFTW3 (optional) - some of the helper tools perform Fourier transformations fftw

Linux and Mac OS

git clone https://github.com/ggerganov/kbd-audio
cd kbd-audio
git submodule update --init
mkdir build && cd build
cmake ..
make

Windows

(todo, PRs welcome)

Tools

Short summary of the available tools. If the status of the tool is not stable, expect problems and non-optimal results.

Name Type Status
record text stable
record-full text stable
play text stable
play-full text stable
view-gui gui stable
view-full-gui gui stable
keytap text stable
keytap-gui gui stable
keytap2 text development
keytap2-gui gui development
- extra -
guess_qp text experiment
guess_qp2 text experiment
key_detector text experiment
scale text experiment
subreak text experiment
key_average_gui gui experiment

Tool details

  • record-full

    Record audio to a raw binary file on disk

    ./record-full output.kbd [-cN]
    

  • play-full

    Playback a recording captured via the record-full tool

    ./play-full input.kbd [-pN]
    

  • record

    Record audio only while typing. Useful for collecting training data for keytap

    ./record output.kbd [-cN]
    

  • play

    Playback a recording created via the record tool

    ./play input.kbd [-pN]
    

  • keytap

    Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool.

    ./keytap input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-pF] [-tF]
    

  • keytap-gui

    Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool. GUI version.

    ./keytap-gui input0.kbd [input1.kbd] [input2.kbd] ... [-cN]
    

    **Live demo (WebAssembly threads required) **

    keytap-gui


  • keytap2-gui (work in progress)

    Detect pressed keys via microphone audio capture. Uses statistical information (n-gram frequencies) about the language. No training data is required. The 'recording.kbd' input file has to be generated via the record-full tool and contains the audio data that will be analyzed. The 'n-gram.txt' file has to contain n-gram probabilities for the corresponding language.

    ./keytap2-gui recording.kbd n-gram.txt
    

    keytap2-gui


  • view-full-gui

    Visualize waveforms recorded with the record-full tool. Can also playback the audio data.

    ./view-full-gui input.kbd
    

    view-full-gui


  • view-gui

    Visualize training data recorded with the record tool. Can also playback the audio data.

    ./view-gui input.kbd
    

    view-full-gui


Feedback

Any feedback about the performance of the tools is highly appreciated. Please drop a comment here.

 

Sursa: https://github.com/ggerganov/kbd-audio

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...