Search…

Cryptography fundamentals

In this series (16 parts)
  1. How attackers think: the attacker mindset
  2. Networking fundamentals for security
  3. Cryptography fundamentals
  4. Public key infrastructure and certificates
  5. Authentication and authorization
  6. Web application security: OWASP Top 10
  7. Network attacks and defenses
  8. Linux privilege escalation
  9. Windows security fundamentals
  10. Malware types and analysis basics
  11. Reconnaissance and OSINT
  12. Exploitation basics and CVEs
  13. Post-exploitation and persistence
  14. Defensive security: hardening and monitoring
  15. Incident response
  16. CTF skills and practice labs

Cryptography is the math that keeps secrets. Every time you log into a website, send a message, or make a payment, cryptography protects the data from being read or tampered with. You do not need to implement these algorithms, but you need to understand what they do, when to use them, and how they fail.

Prerequisites

You should understand networking fundamentals to see where encryption fits in the protocol stack. For the mathematical foundations, probability and information theory provide the theoretical basis.

The three goals of cryptography

  1. Confidentiality: only authorized parties can read the data
  2. Integrity: data has not been tampered with
  3. Authentication: you are talking to who you think you are talking to

Different tools serve different goals:

ToolConfidentialityIntegrityAuthentication
Symmetric encryption (AES)
Asymmetric encryption (RSA)
Hashing (SHA-256)
HMAC
Digital signatures
TLS/SSL

Symmetric encryption

One key encrypts and decrypts. Both parties must share the same key.

graph LR
  A[Plaintext] -->|Encrypt with Key K| B[Ciphertext]
  B -->|Decrypt with Key K| C[Plaintext]
  style A fill:#81c784,stroke:#388e3c,color:#000
  style B fill:#ef5350,stroke:#c62828,color:#fff
  style C fill:#81c784,stroke:#388e3c,color:#000

AES (Advanced Encryption Standard) is the standard. It comes in three key sizes: 128, 192, and 256 bits. AES-256 is considered secure against all known attacks, including theoretical quantum attacks.

# Encrypt a file with AES-256 using OpenSSL
openssl enc -aes-256-cbc -salt -pbkdf2 -in secret.txt -out secret.enc

Output:

enter AES-256-CBC encryption password:
Verifying - enter AES-256-CBC encryption password:
# Decrypt
openssl enc -d -aes-256-cbc -pbkdf2 -in secret.enc -out decrypted.txt

The key problem: how do two parties agree on a shared key without an eavesdropper learning it? This is solved by asymmetric encryption and key exchange.

Asymmetric encryption

Two keys: a public key (shared with everyone) and a private key (kept secret).

  • Encrypt with the public key, decrypt with the private key (confidentiality)
  • Sign with the private key, verify with the public key (authentication)
graph LR
  subgraph Encryption
      A1[Plaintext] -->|Encrypt with Public Key| B1[Ciphertext]
      B1 -->|Decrypt with Private Key| C1[Plaintext]
  end
  subgraph Signing
      A2[Message] -->|Sign with Private Key| B2[Signature]
      B2 -->|Verify with Public Key| C2["Valid ✓ / Invalid ✗"]
  end

RSA is the most widely known asymmetric algorithm. Key sizes of 2048 or 4096 bits are standard. RSA is slower than AES, so in practice it is used to exchange a symmetric key, and AES handles the bulk encryption. This is exactly what TLS does.

# Generate an RSA key pair
openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -pubout -out public.pem

# Encrypt with the public key
echo "secret message" | openssl rsautl -encrypt -pubin -inkey public.pem -out encrypted.bin

# Decrypt with the private key
openssl rsautl -decrypt -inkey private.pem -in encrypted.bin

Output:

secret message

Key exchange: Diffie-Hellman

Diffie-Hellman lets two parties create a shared secret over an insecure channel without ever transmitting the secret itself. The math relies on the difficulty of the discrete logarithm problem.

Simplified version:

  1. Alice and Bob agree on a public base g and modulus p
  2. Alice picks a secret a, computes A = g^a mod p, sends A to Bob
  3. Bob picks a secret b, computes B = g^b mod p, sends B to Alice
  4. Alice computes s = B^a mod p
  5. Bob computes s = A^b mod p
  6. Both arrive at the same shared secret s

An eavesdropper sees g, p, A, and B, but cannot compute s without knowing a or b.

Modern TLS uses Elliptic Curve Diffie-Hellman (ECDHE), which achieves the same security with smaller keys.

Hashing

A hash function takes any input and produces a fixed-size output (the hash or digest). It is a one-way function: you cannot reverse a hash to get the original input.

Properties of a good hash:

  • Deterministic: same input always produces the same hash
  • Fast to compute
  • Collision resistant: hard to find two different inputs with the same hash
  • Avalanche effect: a small change in input produces a completely different hash
# SHA-256 hash
echo -n "hello" | sha256sum

Output:

2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824  -
# Change one character
echo -n "Hello" | sha256sum

Output:

185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969  -

Completely different hash from changing one letter. This is the avalanche effect.

Common hash algorithms

AlgorithmOutput sizeStatusUse case
MD5128 bitsBrokenLegacy checksums only
SHA-1160 bitsWeakBeing phased out
SHA-256256 bitsSecureGeneral purpose, TLS, blockchain
SHA-3VariableSecureNewer alternative to SHA-2
bcrypt184 bitsSecurePassword hashing
Argon2VariableSecurePassword hashing (recommended)

Example 1: Trace a TLS handshake step by step

TLS (Transport Layer Security) combines everything above: key exchange, symmetric encryption, hashing, and certificates. Let’s trace what happens when you connect to a website.

openssl s_client -connect example.com:443 -brief

Output:

CONNECTION ESTABLISHED
Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
Peer certificate: CN = example.com
Hash used: SHA256
Signature type: ECDSA
Verification: OK

Here is what happened step by step:

Step 1: Client Hello Your browser sends: “I support TLS 1.3 and 1.2, these cipher suites, and here is a random number.”

Step 2: Server Hello The server responds: “Let’s use TLS 1.3 with AES-256-GCM and SHA-384. Here is my random number and my certificate.”

Step 3: Certificate verification Your browser checks:

  • Is the certificate valid (not expired)?
  • Was it issued by a trusted CA (Certificate Authority)?
  • Does the domain name match?

Step 4: Key exchange (ECDHE) Client and server perform Elliptic Curve Diffie-Hellman to agree on a shared secret. This happens without transmitting the secret.

Step 5: Derive session keys From the shared secret and random numbers, both sides derive:

  • An encryption key for client-to-server
  • An encryption key for server-to-client
  • An integrity key for each direction

Step 6: Encrypted communication begins All further data is encrypted with AES-256-GCM using the derived keys.

See the full handshake in verbose mode:

openssl s_client -connect example.com:443 -state 2>&1 | head -30

Output:

CONNECTED(00000003)
SSL_connect:before SSL initialization
SSL_connect:SSLv3/TLS write client hello
SSL_connect:SSLv3/TLS read server hello
depth=2 C = US, O = DigiCert Inc, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
verify return:1
depth=0 CN = example.com
verify return:1
SSL_connect:SSLv3/TLS read server certificate
SSL_connect:SSLv3/TLS read server key exchange
SSL_connect:SSLv3/TLS write client key exchange
SSL_connect:SSLv3/TLS write change cipher spec
SSL_connect:SSLv3/TLS write finished
SSL_connect:SSLv3/TLS read change cipher spec
SSL_connect:SSLv3/TLS read finished

Example 2: Why MD5 is broken

MD5 was widely used for file integrity and password hashing. It is broken because researchers can create collisions: two different inputs that produce the same hash.

# Demonstrate that MD5 produces a hash
echo -n "hello" | md5sum

Output:

5d41402abc4b2a76b9719d911017c592  -

The problem is not that MD5 can be reversed (it cannot). The problem is collisions. In 2004, researchers showed they could create two different files with the same MD5 hash in seconds. In 2012, the Flame malware used an MD5 collision to forge a Windows Update certificate.

Practical impact:

# Both files have the same MD5 hash but different content
# (This is a conceptual demonstration - actual collision files
# require specific construction)
md5sum file_a.bin file_b.bin
# SHA-256 catches the difference
sha256sum file_a.bin file_b.bin

The SHA-256 hashes would be completely different, because SHA-256 is collision-resistant.

Rules:

  • Never use MD5 for security (password hashing, integrity verification of sensitive files)
  • Never use MD5 for file integrity if an adversary could tamper with the file
  • SHA-256 is the safe default for general hashing
  • For passwords, use bcrypt or Argon2 (they are deliberately slow, see Authentication)
# Verify a file download with SHA-256 (the right way)
sha256sum downloaded-file.iso
# Compare the output to the hash published on the website

What comes next

The next article covers Public key infrastructure and certificates, where you will learn how the trust chain works, how to inspect certificates, and what happens when it breaks.

For the mathematics behind cryptographic security, see probability fundamentals and information theory.

Start typing to search across all content
navigate Enter open Esc close