55 lines
2.2 KiB
Plaintext
55 lines
2.2 KiB
Plaintext
Environment
|
|
-- each 'environment' follows a single roundabout and track id (recording of that roundabout)
|
|
-- on reset, the environment we will use changes the vehicle to control while having the other agents follow their true data (expert controller)
|
|
---- Note this can be problematic as it can lead to vehicles behind you crashing into you
|
|
|
|
TRAINING
|
|
---------
|
|
1. Load pre-trained massive set of transitions
|
|
-- For all roundabouts
|
|
-- For all tracks
|
|
-- For all vehicles
|
|
-- For all valid timesteps
|
|
-- Rasterized state (incl. path), action
|
|
|
|
2. HGAIL
|
|
-- For each epoch
|
|
-- INSTANTIATE A NEW ENVIRONMENT (Roundabout + Track) w/ randomized agent, from set of all expert environments
|
|
-- Train discriminator off training data + yielded low-level transitions in replay buffer
|
|
-- Train generator off yielded high-level transitions + summed low-level discriminator rewards
|
|
|
|
TESTING
|
|
----------
|
|
1. Save average vehicle velocities for all expert vehicles (loop roundabout + track + vehicle, average over time)
|
|
|
|
2. Run test suite for: expert, BC, GAIL, RAIL, HGAIL, (and hopefully HRAIL)
|
|
-- For all roundabouts, tracks
|
|
-- Get expert velocities for track
|
|
-- Simulate incrementing agent environment (e.g. on reset, agent +=1)
|
|
-- Store low-level true joint states, actions, and controlled vehicle index
|
|
-- Per-vehicle statistics (v_all, v_mean, v_shortfall, a_all, jerk_all, n_collisions, T)
|
|
-- Aggregate statistics + joint
|
|
|
|
Problems
|
|
-----------
|
|
Should train without stopping for collisions, however when doing so, end up with policy that always takes decelerate option
|
|
-- It seems safe at the start of each vehicles sim, but actually it isn't since a car will spawn and hit it
|
|
Solutions:
|
|
-- Hold cars from spawning if their spawn location is full
|
|
-- Start simulations a few seconds later (after cars clear their spawn places) <- Preferred
|
|
|
|
Test could run indefinitely if stop_on_collision is off
|
|
Solution:
|
|
-- Set maximum episode length in intersimple
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Save massive set of transition raw states beforehand (1 from training, but with raw states)
|
|
# -- For all roundabouts, tracks
|
|
# -- For all vehicles, steps
|
|
# -- Raw vehicle state, action |