Verification System
How the automated testing system works
DALT.PHP includes an automated verification system that checks if you've correctly fixed challenge bugs.
Overview
The verification system:
- Runs automated tests on your code
- Checks for specific fixes and patterns
- Provides helpful hints when tests fail
- Tracks your progress
- Gives instant feedback
How It Works
Architecture
Challenge Directory
├── README.md # Challenge description
├── tests.php # Test specification
├── routes/ # Broken files to copy
└── Http/controllers/ # Broken controllersVerification Flow
- You fix the bug - Edit files to solve the challenge
- Run verification - Execute
php artisan verify <challenge> - Tests execute - System runs automated checks
- Get feedback - See which tests passed/failed
- Iterate - Fix remaining issues and re-verify
Running Verification
Command Line
php artisan verify <challenge-name>Available challenges:
broken-routingbroken-middlewarebroken-authbroken-databasebroken-session
Example:
php artisan verify broken-routingWeb Interface
- Visit
http://localhost:8000/learn - Click on a challenge
- Click "Run Verification" button
- View results in browser
Verification Output
Success Example
╔══════════════════════════════════════════════════════════════╗
║ DALT Challenge Verification System ║
╚══════════════════════════════════════════════════════════════╝
Verifying: broken-routing
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✓ Route get /posts/create exists
✓ Route get /posts/{id}/edit exists
✓ Route order correct: specific before generic
✓ No problematic code found
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Results: 4/4 tests passed
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ All tests passed! Challenge complete!
Great job! You've successfully fixed this challenge.
Check storage/logs/challenges.log for your progress.Failure Example
╔══════════════════════════════════════════════════════════════╗
║ DALT Challenge Verification System ║
╚══════════════════════════════════════════════════════════════╝
Verifying: broken-routing
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✓ Route get /posts/create exists
✗ Route get /posts/{id}/edit not found
💡 Hint: Uncomment the /posts/{id}/edit route in routes/routes.php
✓ Route order correct: specific before generic
✗ File still contains: // $router->get('/posts/{id}/edit'
💡 Hint: Remove the // comment from the edit route
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Results: 2/4 tests passed
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
❌ 2 of 4 tests failed. Keep debugging!
💡 Next step: Uncomment the /posts/{id}/edit route in routes/routes.php
Keep debugging! Check the challenge README for more hints.Test Types
route_exists
Checks if a route is registered in routes/routes.php.
'route_create_exists' => [
'type' => 'route_exists',
'route' => '/posts/create',
'method' => 'get',
'hint' => 'Make sure /posts/create route is registered'
]What it checks:
- Route pattern exists
- HTTP method matches
- Route is not commented out
route_order
Checks if routes are in correct order (specific before generic).
'route_order_correct' => [
'type' => 'route_order',
'specific' => '/posts/create',
'generic' => '/posts/{id}',
'hint' => 'Move /posts/create BEFORE /posts/{id}'
]What it checks:
- Both routes exist
- Specific route appears before generic route in file
file_contains
Checks if a file contains specific code.
'uses_password_verify' => [
'type' => 'file_contains',
'file' => 'framework/Core/Authenticator.php',
'search' => 'password_verify',
'hint' => 'Use password_verify() to check passwords'
]What it checks:
- File exists
- Contains exact search string
file_not_contains
Checks if a file does NOT contain specific code (e.g., bugs).
'no_plain_comparison' => [
'type' => 'file_not_contains',
'file' => 'framework/Core/Authenticator.php',
'search' => '$password ==',
'hint' => 'Remove plain text password comparison'
]What it checks:
- File exists
- Does NOT contain search string
session_key
Checks if correct session key is used.
'correct_session_key' => [
'type' => 'session_key',
'file' => 'framework/Core/Middleware/Auth.php',
'key' => 'user',
'hint' => "Use \$_SESSION['user'] instead"
]What it checks:
- File contains
$_SESSION['key']usage - Correct key name is used
function_call
Checks if a specific function is called.
'uses_hash_equals' => [
'type' => 'function_call',
'file' => 'framework/Core/Middleware/Csrf.php',
'function' => 'hash_equals',
'hint' => 'Use hash_equals() for timing-safe comparison'
]What it checks:
- File contains function name
- Function is being called
Creating Custom Tests
Test File Structure
Create course/challenges/your-challenge/tests.php:
<?php
/**
* Your Challenge - Test Specification
*/
return [
'test_name_1' => [
'type' => 'route_exists',
'route' => '/your/route',
'method' => 'get',
'hint' => 'Helpful hint for students'
],
'test_name_2' => [
'type' => 'file_contains',
'file' => 'path/to/file.php',
'search' => 'expected_code',
'hint' => 'Another helpful hint'
],
// Add more tests...
];Test Configuration
Each test requires:
Required fields:
type- Test type (see Test Types above)- Type-specific fields (route, file, search, etc.)
Optional fields:
hint- Shown when test fails
Example: Custom Challenge
Create Challenge Directory
mkdir -p course/challenges/my-challengeCreate Test File
Create course/challenges/my-challenge/tests.php:
<?php
return [
'controller_exists' => [
'type' => 'file_contains',
'file' => 'Http/controllers/api/posts.php',
'search' => 'class PostsController',
'hint' => 'Create PostsController class'
],
'returns_json' => [
'type' => 'file_contains',
'file' => 'Http/controllers/api/posts.php',
'search' => 'header(\'Content-Type: application/json\')',
'hint' => 'Set JSON content type header'
],
'route_registered' => [
'type' => 'route_exists',
'route' => '/api/posts',
'method' => 'get',
'hint' => 'Register /api/posts route'
]
];Create README
Create course/challenges/my-challenge/README.md with challenge description.
Test Verification
php artisan verify my-challengeProgress Tracking
Log File
Verification results are logged to storage/logs/challenges.log:
[2024-03-15 10:30:45] broken-routing - pass (4/4)
[2024-03-15 11:15:22] broken-middleware - fail (2/4)
[2024-03-15 11:45:10] broken-middleware - pass (4/4)
[2024-03-15 14:20:33] broken-auth - pass (3/3)Format: [timestamp] challenge - status (passed/total)
View Progress
# View all attempts
cat storage/logs/challenges.log
# View specific challenge
grep "broken-routing" storage/logs/challenges.log
# Count completions
grep "pass" storage/logs/challenges.log | wc -lHint System
How Hints Work
- Test fails - Verification detects issue
- Hint provided - Specific guidance shown
- Student fixes - Apply suggested fix
- Re-verify - Run verification again
Writing Good Hints
Bad hint:
'hint' => 'Fix the code'Good hint:
'hint' => 'Use password_verify() instead of == for password comparison'Great hint:
'hint' => 'Replace $password == $user[\'password\'] with password_verify($password, $user[\'password\'])'Hint Best Practices
- Be specific - Tell exactly what to change
- Show location - Mention file and line if possible
- Explain why - Brief reason for the fix
- Progressive - Start general, get specific on retry
Advanced Features
Custom Test Types
Extend ChallengeVerifier class to add new test types:
// In framework/Core/ChallengeVerifier.php
private function testCustomType(array $config): array
{
// Your custom test logic
return [
'passed' => $testPassed,
'message' => 'Test result message',
'hint' => 'Helpful hint if failed'
];
}Then use in tests.php:
'my_custom_test' => [
'type' => 'custom_type',
'param1' => 'value1',
'hint' => 'Custom hint'
]Multiple File Checks
Check multiple files in one test:
'all_files_updated' => [
'type' => 'file_contains',
'file' => 'Http/controllers/auth/login.php',
'search' => 'password_verify',
'hint' => 'Update login controller'
],
'middleware_updated' => [
'type' => 'file_contains',
'file' => 'framework/Core/Middleware/Auth.php',
'search' => '$_SESSION[\'user\']',
'hint' => 'Update auth middleware'
]Regex Patterns
Use regex for flexible matching:
'uses_prepared_statement' => [
'type' => 'file_contains',
'file' => 'framework/Core/Database.php',
'search' => '$this->statement->execute($params)',
'hint' => 'Pass $params to execute() method'
]Troubleshooting
Verification Not Running
Problem: php artisan verify shows error.
Check:
- Challenge directory exists
- tests.php file exists
- File paths are correct
# Check challenge exists
ls course/challenges/broken-routing/
# Check tests file
cat course/challenges/broken-routing/tests.phpTests Always Fail
Problem: Tests fail even after fixing.
Debug:
// Add to tests.php temporarily
dd(file_get_contents(base_path('routes/routes.php')));False Positives
Problem: Test passes but code is still broken.
Solution: Make tests more specific:
// Too broad
'search' => 'password'
// More specific
'search' => 'password_verify($password, $hash)'Hints Not Showing
Problem: No hints displayed on failure.
Check: Ensure hint key exists in test config:
'my_test' => [
'type' => 'file_contains',
'file' => 'path/to/file.php',
'search' => 'code',
'hint' => 'Add this hint!' // Required for hints
]Best Practices
For Students
- Read challenge README first - Understand the bug
- Run verification early - See what's expected
- Fix one test at a time - Incremental progress
- Read hints carefully - They're specific to your issue
- Re-verify often - Get immediate feedback
For Challenge Creators
- Test incrementally - One concept per test
- Provide clear hints - Specific, actionable guidance
- Check edge cases - Test both correct and incorrect solutions
- Document expected behavior - Clear README
- Test your tests - Verify they work correctly
Integration with Learning
Challenge Workflow
- Study lesson - Learn the concept
- Read challenge - Understand the bug
- Copy broken files - Setup challenge
- Debug code - Find and fix issues
- Run verification - Check solution
- Review feedback - Learn from results
Verification in Curriculum
Lesson 02: Routing
↓
Challenge: Broken Routing
↓
Verification: 4 tests
↓
Pass → Next Lesson
Fail → Review + RetryAPI Reference
ChallengeVerifier Class
$verifier = new Core\ChallengeVerifier('course/challenges/broken-routing');
$result = $verifier->verify();Returns:
[
'status' => 'pass|fail|error',
'message' => 'Result message',
'hint' => 'Next step hint',
'passed' => 3,
'failed' => 1,
'total' => 4,
'results' => [
[
'name' => 'test_name',
'passed' => true,
'message' => 'Test message',
'hint' => 'Test hint'
],
// More results...
]
]Static Methods
// Log verification result
Core\ChallengeVerifier::logResult($challengeName, $result);Next Steps
- CLI Commands - Run verification from command line
- Challenges - Try the challenges
- Contributing - Create your own challenges