Compare commits
27 Commits
e0d205bbd7
...
idm-overri
| Author | SHA1 | Date | |
|---|---|---|---|
|
3aaf252dbe | ||
|
|
779a0ea89f | ||
|
|
3a09a6eb7d | ||
|
|
9c9ee8f21b | ||
|
|
f814516072 | ||
|
|
a9feec4f38 | ||
|
|
3d3b3d510a | ||
|
|
3ac9465997 | ||
|
|
597b9af5d4 | ||
|
|
37f44605d2 | ||
|
|
4076b0361a | ||
|
|
fc04f8e9ee | ||
|
|
f5f1c24f45 | ||
|
|
f93e130498 | ||
|
|
2965dc9982 | ||
|
|
fa0e20998d | ||
|
|
57a42f70ec | ||
|
|
5f6ad37c37 | ||
|
|
e602aa0641 | ||
|
|
5ada1cc543 | ||
|
|
e28459a168 | ||
|
|
b94344214b | ||
|
|
6d867466c6 | ||
|
|
8e12996dfe | ||
|
|
a3280893af | ||
|
|
9c3cb4fb55 | ||
|
|
a1db6aa553 |
69
README.md
69
README.md
@@ -1,10 +1,22 @@
|
||||
# InteractionImitation
|
||||
Imitation Learning with the INTERACTION Dataset
|
||||
Imitation Learning with the [Interaction Dataset](https://interaction-dataset.com/) via the [InteractionSimulator](https://github.com/sisl/InteractionSimulator) gym environments.
|
||||
|
||||
Code for "[SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments](https://arxiv.org/abs/2204.01922)".
|
||||
If you find this repository useful, please cite the paper:
|
||||
|
||||
```
|
||||
@article{jamgochian2022shail,
|
||||
author = {Arec Jamgochian and Etienne Buehrle and Johannes Fischer and Mykel J. Kochenderfer},
|
||||
title = {{SHAIL}: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments},
|
||||
journal = {arXiv:2204.01922 [cs]},
|
||||
year = {2022}
|
||||
}
|
||||
```
|
||||
|
||||
## Getting started
|
||||
Clone InteractionSimulator and pip install the module.
|
||||
Clone the `InteractionSimulator` with the `shail` tag and pip install the module.
|
||||
```
|
||||
git clone https://github.com/sisl/InteractionSimulator.git
|
||||
git clone --branch shail https://github.com/sisl/InteractionSimulator.git
|
||||
cd InteractionSimulator
|
||||
pip install -e .
|
||||
cd ..
|
||||
@@ -19,53 +31,28 @@ The INTERACTION dataset contains a two folders which should be copied into a fol
|
||||
- the contents of `recorded_trackfiles` should be copied to `./InteractionSimulator/datasets/trackfiles`
|
||||
- the contents of `maps` should be copied to `./InteractionSimulator/datasets/maps`
|
||||
|
||||
## Processing, saving, and loading expert demos
|
||||
Once the repository has been set up, you can process and save expert track demonstrations with:
|
||||
## Processing and saving expert demos
|
||||
Once the repository has been set up, you need to generate two separate sets of expert demos for tracks 0-4. The first command generates true joint and individual states and actions necessary for evaluating, saving them in `expert_data/`. The second command generates trajectory rollouts according to individual agent observations, which is later used as expert data for the learning models.
|
||||
```
|
||||
python src/expert_data.py --loc [LOCNUM] --track [TRACKNUM]
|
||||
```
|
||||
You can (and should) process all tracks at once at location 0 with:
|
||||
```
|
||||
python src/expert_data.py --all-tracks
|
||||
```
|
||||
|
||||
You can then train a default behavior cloning policy with the following. Be sure to check help for main.py for running options.
|
||||
```
|
||||
python src/main.py --train
|
||||
```
|
||||
You can run tensorboard by running the following and opening `localhost:6006` (or alternatively port-forwarding 6006 from the remote server)
|
||||
```
|
||||
tensorboard --logdir output/
|
||||
```
|
||||
You can then test the learned policy with the following, and see the animation file in `output/`:
|
||||
```
|
||||
python src/main.py --test
|
||||
python -m src.expert --locs='[DR_USA_Roundabout_FT]' --tracks='[0,1,2,3,4]'
|
||||
python -m intersimple-expert-rollout-setobs2 --tracks='[0,1,2,3,4]'
|
||||
```
|
||||
|
||||
|
||||
You can load the experts actions manually
|
||||
```
|
||||
from src import expert_data
|
||||
observations, actions = expert_data.load_expert_data(loc = [LOCNUM], track = [TRACKNUM])
|
||||
for (s, a) in zip (observations, actions):
|
||||
# do some imitation learning
|
||||
```
|
||||
## Tuning hyperparameters and training finalized models
|
||||
To tune models, we use `ray[tune]` grid searches. You can run see the commands we used to train in the top half of `train_models.sh`, as well as the hyperparameters we search over in `bc-experiment.py`, `gail-experiment.py`, and `shail-experiment.py`. After training the models, configurations get saved in `best_configs/` (the best SHAIL confg gets copied to a HAIL config, with the appropriate environment parameters changed for ablation). However, upon manual inspection of the training runs, we note some better performance than the automatically-set configs at earlier epochs, so we adjust the `best_configs` manually.
|
||||
|
||||
After the `best_configs/` are set, we rerun each configuration with multiple seeds. The commands to do so are in the bottom half of `train_models.sh`. This saves different learned policy files to `test_policies/`.
|
||||
|
||||
|
||||
## Evaluating models
|
||||
To evaluate the learned policies, we rerun each model in particular setting, evaluate all our metrics, and average over different trained model seeds. The commands to do so are in `evaluate_models.sh`.
|
||||
|
||||
|
||||
## Package Structure
|
||||
```
|
||||
InteractionImitation
|
||||
|- demos
|
||||
|- algorithms
|
||||
|- BC
|
||||
|- AdVIL
|
||||
|- nets
|
||||
|- Encoder
|
||||
|- DeepSet
|
||||
|- Decoder
|
||||
|- policies
|
||||
|- discriminators
|
||||
|- demo_generators
|
||||
|- TODO
|
||||
```
|
||||
|
||||
## Type Definitions
|
||||
|
||||
@@ -55,6 +55,7 @@ def training_function(config):
|
||||
),
|
||||
check_collisions=True,
|
||||
stop_on_collision=config['trainenv']['stop_on_collision'],
|
||||
use_idm=config['trainenv']['use_idm'],
|
||||
), collision_distance=6, collision_penalty=100),
|
||||
lambda obs: (obs - obs_min) / (obs_max - obs_min + 1e-10)
|
||||
)) for _ in range(60)]
|
||||
@@ -68,6 +69,8 @@ def training_function(config):
|
||||
),
|
||||
check_collisions=True,
|
||||
stop_on_collision=config['trainenv']['stop_on_collision'],
|
||||
use_idm=config['trainenv']['use_idm'],
|
||||
track=track,
|
||||
), collision_distance=6, collision_penalty=100),
|
||||
lambda obs: (obs - obs_min) / (obs_max - obs_min + 1e-10)
|
||||
)) for _ in range(15)] for track in range(4)],[])
|
||||
@@ -142,7 +145,7 @@ if __name__ == '__main__':
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--train', choices=['A', 'B'])
|
||||
parser.add_argument('--epochs', type=int, default=1000)
|
||||
parser.add_argument('--epochs', type=int, default=500)
|
||||
parser.add_argument('--test', type=str, help='path to config file to run final training on')
|
||||
parser.add_argument('--test_seeds', type=int, default=5)
|
||||
parser.add_argument('--test_cpus', type=int, help='number of cpus available to split test seed training over')
|
||||
@@ -159,13 +162,14 @@ if __name__ == '__main__':
|
||||
'experiment': args.train,
|
||||
'trainenv': {
|
||||
'stop_on_collision': False,
|
||||
'use_idm':True,
|
||||
},
|
||||
'policy': {
|
||||
'learning_rate': 3e-4,
|
||||
'learning_rate_decay': tune.grid_search([0.001, 1.0]),
|
||||
'hidden_layer_size': tune.grid_search([10, 20, 40, 80]),
|
||||
'n_hidden_layers': tune.grid_search([2, 3, 4]),
|
||||
'activation':0,
|
||||
'learning_rate_decay': tune.grid_search([0.999, 1.0]),
|
||||
'hidden_layer_size': tune.grid_search([10, 20, 40]),
|
||||
'n_hidden_layers': tune.grid_search([2, 3]),
|
||||
'activation':tune.grid_search([0, 1]),
|
||||
},
|
||||
'train_epochs': args.epochs,
|
||||
'seed': 0,
|
||||
@@ -210,3 +214,5 @@ if __name__ == '__main__':
|
||||
check_dir = analysis._checkpoints[i]['logdir']
|
||||
shutil.copyfile(os.path.join(check_dir,'policy_final.pt'),
|
||||
os.path.join(savepath, f'policy_seed{s}.pt'))
|
||||
shutil.copyfile(os.path.join(check_dir,'params.json'),
|
||||
os.path.join(savepath, 'config.json')) # copy config automatically
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
"stop_on_collision": false,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
"stop_on_collision": false,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"delta": 0.01,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 2,
|
||||
"activation": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.0001,
|
||||
"weight_decay": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 4,
|
||||
"n_hidden_layers_global": 1,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"delta": 0.01,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 2,
|
||||
"activation": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.0001,
|
||||
"weight_decay": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 4,
|
||||
"n_hidden_layers_global": 1,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
"stop_on_collision": false,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false
|
||||
"stop_on_collision": false,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 10,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 10,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 20,
|
||||
"n_hidden_layers": 4,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 20,
|
||||
"n_hidden_layers": 4,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -3,7 +3,8 @@
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
"abort_unsafe_collision_method": null,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": null,
|
||||
"abort_unsafe_collision_method": null
|
||||
"abort_unsafe_collision_method": null,
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 10,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 10,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 40,
|
||||
"n_hidden_layers": 3,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "A",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 20,
|
||||
"n_hidden_layers": 4,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"experiment": "B",
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
"learning_rate_decay": 1.0,
|
||||
"clip_ratio": 0.2,
|
||||
"iterations_per_epoch": 100,
|
||||
"hidden_layer_size": 20,
|
||||
"n_hidden_layers": 4,
|
||||
"activation": 0,
|
||||
"option": 0
|
||||
},
|
||||
"value": {
|
||||
"learning_rate": 0.001,
|
||||
"iterations_per_epoch": 1000
|
||||
},
|
||||
"discriminator": {
|
||||
"learning_rate": 0.001,
|
||||
"weight_decay": 0.0001,
|
||||
"iterations_per_epoch": 100,
|
||||
"n_hidden_layers_element": 3,
|
||||
"n_hidden_layers_global": 2,
|
||||
"hidden_layer_size": 10,
|
||||
"activation": 0
|
||||
},
|
||||
"train_epochs": 100,
|
||||
"seed": 0
|
||||
}
|
||||
@@ -3,7 +3,8 @@
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
"abort_unsafe_collision_method": "circle",
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
"trainenv": {
|
||||
"stop_on_collision": false,
|
||||
"safe_actions_collision_method": "circle",
|
||||
"abort_unsafe_collision_method": "circle"
|
||||
"abort_unsafe_collision_method": "circle",
|
||||
"use_idm": true
|
||||
},
|
||||
"policy": {
|
||||
"learning_rate": 0.0003,
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,42 +0,0 @@
|
||||
{
|
||||
ego_encoder: {
|
||||
input_dim: 5, // number of state vars
|
||||
hidden_n: 0,
|
||||
hidden_dim: 5,
|
||||
output_dim: 5
|
||||
},
|
||||
deepsets: {
|
||||
input_dim: 6, // number of relative state vars for others
|
||||
phi: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 20,
|
||||
},
|
||||
latent_dim: 20,
|
||||
rho: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 10,
|
||||
},
|
||||
output_dim: 10
|
||||
},
|
||||
path_encoder: {
|
||||
input_dim: 40, // 2 * path length for (x,y) coordinates
|
||||
hidden_n: 0,
|
||||
hidden_dim: 20,
|
||||
output_dim: 10,
|
||||
},
|
||||
head: {
|
||||
input_dim: 0, // computed in policy constructor
|
||||
hidden_n: 3,
|
||||
hidden_dim: 50,
|
||||
output_dim: 1, // number of outputs e.g. number of actions, or just one
|
||||
final_activation: 'sigmoid',
|
||||
},
|
||||
optim: {
|
||||
optimizer: 'adam',
|
||||
lr: 1e-3,
|
||||
weight_decay: 0.1,
|
||||
},
|
||||
train_epochs: 200,
|
||||
train_batch_size: 32,
|
||||
loss: 'huber',
|
||||
}
|
||||
@@ -1,85 +0,0 @@
|
||||
{
|
||||
policy_net: {
|
||||
ego_encoder: {
|
||||
input_dim: 5, // number of state vars
|
||||
hidden_n: 0,
|
||||
hidden_dim: 5,
|
||||
output_dim: 5
|
||||
},
|
||||
deepsets: {
|
||||
input_dim: 6, // number of relative state vars for others
|
||||
phi: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 20,
|
||||
},
|
||||
latent_dim: 20,
|
||||
rho: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 10,
|
||||
},
|
||||
output_dim: 10
|
||||
},
|
||||
path_encoder: {
|
||||
input_dim: 40, // 2 * path length for (x,y) coordinates
|
||||
hidden_n: 0,
|
||||
hidden_dim: 20,
|
||||
output_dim: 10,
|
||||
},
|
||||
head: {
|
||||
input_dim: 0, // computed in policy constructor
|
||||
hidden_n: 3,
|
||||
hidden_dim: 50,
|
||||
output_dim: 1, // number of outputs e.g. number of actions, or just one
|
||||
final_activation: 'sigmoid',
|
||||
},
|
||||
},
|
||||
value_net: {
|
||||
ego_encoder: {
|
||||
input_dim: 5, // number of state vars
|
||||
hidden_n: 0,
|
||||
hidden_dim: 5,
|
||||
output_dim: 5
|
||||
},
|
||||
deepsets: {
|
||||
input_dim: 6, // number of relative state vars for others
|
||||
phi: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 20,
|
||||
},
|
||||
latent_dim: 20,
|
||||
rho: {
|
||||
hidden_n: 2,
|
||||
hidden_dim: 10,
|
||||
},
|
||||
output_dim: 10
|
||||
},
|
||||
path_encoder: {
|
||||
input_dim: 40, // 2 * path length for (x,y) coordinates
|
||||
hidden_n: 0,
|
||||
hidden_dim: 20,
|
||||
output_dim: 10,
|
||||
},
|
||||
action_dim: 1, // number of actions
|
||||
head: {
|
||||
input_dim: 0, // computed in policy constructor
|
||||
hidden_n: 3,
|
||||
hidden_dim: 50,
|
||||
output_dim: 1, // number of outputs e.g. number of actions, or just one
|
||||
final_activation: 'id',
|
||||
},
|
||||
},
|
||||
policy_optim: {
|
||||
optimizer: 'adam',
|
||||
lr: 1e-3,
|
||||
weight_decay: 0.1,
|
||||
},
|
||||
value_optim: {
|
||||
optimizer: 'adam',
|
||||
lr: 1e-3,
|
||||
weight_decay: 0.1,
|
||||
},
|
||||
train_epochs: 200,
|
||||
train_batch_size: 32,
|
||||
discount: 0.95,
|
||||
clip_grad_norm: 1.,
|
||||
}
|
||||
@@ -12,16 +12,16 @@ def main(method:str='expert', folder:str=None, locations=[(0,0)], skip_running=F
|
||||
policy_kwargs = {}
|
||||
|
||||
if method in ['expert', 'idm']:
|
||||
env, env_kwargs ='NRasterizedRouteIncrementingAgent', {}
|
||||
env, env_kwargs ='NRasterizedRouteIncrementingAgent', {'use_idm':True}
|
||||
elif method in ['bc','gail']:
|
||||
env='NormalizedContinuousEvalEnv'
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000}
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000, 'use_idm':True}
|
||||
elif method in ['hail']:
|
||||
env = 'NormalizedSafeOptionsEvalEnv'
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000, 'safe_actions_collision_method': None, 'abort_unsafe_collision_method': None}
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000, 'safe_actions_collision_method': None, 'abort_unsafe_collision_method': None, 'use_idm':True}
|
||||
elif method in ['shail']:
|
||||
env = 'NormalizedSafeOptionsEvalEnv'
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000}
|
||||
env_kwargs={'stop_on_collision':True, 'max_episode_steps':1000, 'use_idm':True}
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
@@ -35,12 +35,12 @@ def main(method:str='expert', folder:str=None, locations=[(0,0)], skip_running=F
|
||||
print('%i policy files found in %s folder' %(len(files), folder))
|
||||
print('found policy config', config['policy'])
|
||||
|
||||
policy_config = {k: v for k, v in config['policy'].items() if k not in exclude_keys_from_policy_kwargs}
|
||||
policy_config['activation'] = activations[policy_config['activation']]
|
||||
print('final policy config', policy_config)
|
||||
policy_config = {k: v for k, v in config['policy'].items() if k not in exclude_keys_from_policy_kwargs}
|
||||
policy_config['activation'] = activations[policy_config['activation']]
|
||||
print('final policy config', policy_config)
|
||||
|
||||
policy_kwargs.update(policy_config)
|
||||
print('final policy kwargs', policy_kwargs)
|
||||
policy_kwargs.update(policy_config)
|
||||
print('final policy kwargs', policy_kwargs)
|
||||
|
||||
if not skip_running:
|
||||
for policy_file in files:
|
||||
@@ -77,7 +77,7 @@ def latex_print(am, light=False):
|
||||
print('success rate, distance travelled, RWSE_10, |DeltaV|, AccelJSD')
|
||||
if light:
|
||||
if 'rwse_10s' in am.keys():
|
||||
print("%2.1f& %2.1f & %1.2f & %2.1f& "
|
||||
print("%2.1f& %2.1f & %2.1f & %1.2f& "
|
||||
"%0.3f \\\\" %( 100*am['success rate'][0], am['mean travel distance'][0], am['rwse_10s'][0],
|
||||
am['average absolute average velocity'][0],am['acceleration distribution divergence'][0] ))
|
||||
return
|
||||
@@ -88,7 +88,7 @@ def latex_print(am, light=False):
|
||||
return
|
||||
|
||||
print("%2.1f \\scriptstyle\\pm %2.1f & %2.1f \\scriptstyle\\pm %2.1f & "
|
||||
"%1.2f \\scriptstyle\\pm %1.2f & %2.1f \\scriptstyle\\pm %1.1f & "
|
||||
"%2.1f \\scriptstyle\\pm %1.1f & %1.2f \\scriptstyle\\pm %1.2f & "
|
||||
"%0.3f \\scriptstyle\\pm %0.3f \\\\" %( 100*am['success rate'][0], 100*am['success rate'][1],
|
||||
am['mean travel distance'][0] , am['mean travel distance'][1] ,
|
||||
am['rwse_10s'][0] , am['rwse_10s'][1] ,
|
||||
|
||||
@@ -1,19 +1,18 @@
|
||||
# can add --skip_running if you've run the runs before on the saved policies
|
||||
# can add --skip_running if you've already run the saved policies through the test environments and have appropriate
|
||||
# metrics in the out folder. Doing so will generate average metrics quickly.
|
||||
|
||||
# Experiment A
|
||||
python -m eval_experiments
|
||||
python -m eval_experiments --locations='[(0,4)]'
|
||||
python -m eval_experiments --method idm
|
||||
python -m eval_experiments --method idm --locations='[(0,4)]'
|
||||
python -m eval_experiments --method bc --folder='test_policies/bc/expA'
|
||||
python -m eval_experiments --method bc --folder='test_policies/bc/expB' --locations='[(0,4)]'
|
||||
python -m eval_experiments --method gail --folder='test_policies/gail/expA'
|
||||
python -m eval_experiments --method gail --folder='test_policies/gail/expB' --locations='[(0,4)]'
|
||||
python -m eval_experiments --method hail --folder='test_policies/hail/expA'
|
||||
python -m eval_experiments --method hail --folder='test_policies/hail/expB' --locations='[(0,4)]'
|
||||
python -m eval_experiments --method shail --folder='test_policies/shail/expA'
|
||||
python -m eval_experiments --method shail --folder='test_policies/shail/expB' --locations='[(0,4)]'
|
||||
|
||||
python -m eval_experiments --method hail --folder='test_policies/hail-etienne/expA'
|
||||
python -m eval_experiments --method hail --folder='test_policies/hail-etienne/expB' --locations='[(0,4)]'
|
||||
python -m eval_experiments --method shail --folder='test_policies/shail-etienne/expA'
|
||||
python -m eval_experiments --method shail --folder='test_policies/shail-etienne/expB' --locations='[(0,4)]'
|
||||
# Experiment B
|
||||
python -m eval_experiments --locations='[(0,4)]'
|
||||
python -m eval_experiments --method idm --locations='[(0,4)]' --skip_running
|
||||
python -m eval_experiments --method bc --folder='test_policies/bc/expB' --locations='[(0,4)]' --skip_running
|
||||
python -m eval_experiments --method gail --folder='test_policies/gail/expB' --locations='[(0,4)]' --skip_running
|
||||
python -m eval_experiments --method hail --folder='test_policies/hail/expB' --locations='[(0,4)]' --skip_running
|
||||
python -m eval_experiments --method shail --folder='test_policies/shail/expB' --locations='[(0,4)]' --skip_running
|
||||
@@ -1,203 +0,0 @@
|
||||
import json5
|
||||
from functools import partial
|
||||
import os
|
||||
opj = os.path.join
|
||||
|
||||
# set up ray tune
|
||||
import ray
|
||||
from ray import tune
|
||||
from ray.tune import Analysis, ExperimentAnalysis
|
||||
from ray.tune.schedulers import ASHAScheduler
|
||||
from hyperopt import hp
|
||||
from ray.tune.suggest.hyperopt import HyperOptSearch
|
||||
|
||||
# get graphs
|
||||
import intersim
|
||||
from intersim.graphs import ConeVisibilityGraph
|
||||
|
||||
|
||||
from src.main import basestr, main
|
||||
|
||||
def parse_args():
|
||||
"""
|
||||
Parse arguments to main
|
||||
Returns:
|
||||
kwargs: dictionary of arguments:
|
||||
train (bool): whether to run train loop
|
||||
test (bool): whether to run test loop
|
||||
method (str): the method to try for imitation
|
||||
loc (int): the location index of the roundabout
|
||||
config (str): config path
|
||||
seed (int): RNG seed
|
||||
"""
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description='Save Expert Trajectories')
|
||||
parser.add_argument('--loc', default=0, type=int,
|
||||
help='location (default 0)')
|
||||
parser.add_argument("--train", help="train model",
|
||||
action="store_true")
|
||||
parser.add_argument("--ray", help="use ray tune to run multiple experiments",
|
||||
action="store_true")
|
||||
parser.add_argument("--test", help="test model",
|
||||
action="store_true")
|
||||
parser.add_argument("--method", help="modeling method",
|
||||
choices=['bc', 'gail', 'advil', 'vd'], default='bc')
|
||||
parser.add_argument("--config", help="config file path",
|
||||
default=None, type=str)
|
||||
parser.add_argument('--seed', default=0, type=int,
|
||||
help='seed')
|
||||
parser.add_argument('--nframes', default=500, type=int,
|
||||
help='frames for test animation')
|
||||
parser.add_argument('--nsamples', default=200, type=int,
|
||||
help='number of ray samples')
|
||||
parser.add_argument('--graph', action='store_true',
|
||||
help='whether to mask the relative states based on a ConeVisibilityGraph')
|
||||
parser.add_argument('-d', default='./expert_data', type=str,
|
||||
help='data directory')
|
||||
parser.add_argument('-o', default=None, type=str,
|
||||
help='output directory')
|
||||
args = parser.parse_args()
|
||||
kwargs = {
|
||||
'train':args.train,
|
||||
'test':args.test,
|
||||
'method':args.method,
|
||||
'loc':args.loc,
|
||||
'config_path':args.config,
|
||||
'seed':args.seed,
|
||||
'ray':args.ray,
|
||||
'nframes':args.nframes,
|
||||
'nsamples':args.nsamples,
|
||||
'datadir':os.path.abspath(args.d),
|
||||
'graph':None,
|
||||
'outdir': opj('output',args.method,'loc%02i'%(args.loc)),
|
||||
'train_tracks':[0,1,2],
|
||||
'cv_tracks':[3],
|
||||
'test_tracks':[4],
|
||||
}
|
||||
if args.o:
|
||||
kwargs['outdir'] = args.o
|
||||
if args.graph:
|
||||
kwargs['graph'] = ConeVisibilityGraph(r=20, half_angle=120)
|
||||
return kwargs
|
||||
|
||||
def get_full_config(ray_config:dict, method:str)->dict:
|
||||
"""
|
||||
Get full model configuration from ray config and method string
|
||||
Args:
|
||||
ray_config (dict): ray config
|
||||
method (str): method to get full configuration for
|
||||
"""
|
||||
if method == 'bc':
|
||||
from src.bc import bc_config
|
||||
config = bc_config(ray_config)
|
||||
elif method == 'vd':
|
||||
from src.value_dice import vd_config
|
||||
config = vd_config(ray_config)
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return config
|
||||
|
||||
def get_ray_config(method:str)->dict:
|
||||
"""
|
||||
Get configuration for ray based on method.
|
||||
Args:
|
||||
method (str): method to get configuration for
|
||||
Returns:
|
||||
ray_config (dict): configuration for ray
|
||||
"""
|
||||
if method == 'bc':
|
||||
ray_config = {
|
||||
"lr": tune.loguniform(1e-5, 1e-3),
|
||||
"weight_decay": tune.choice([0, 0.1]),
|
||||
"loss": tune.choice(['huber', 'mse']),
|
||||
"train_batch_size": tune.choice([16,32,64]),
|
||||
"deepsets_phi_hidden_n": tune.randint(1,5),
|
||||
"deepsets_phi_hidden_dim": tune.lograndint(8,65),
|
||||
"deepsets_latent_dim": tune.lograndint(8,129),
|
||||
"deepsets_rho_hidden_n": tune.randint(0,3),
|
||||
"deepsets_rho_hidden_dim": tune.lograndint(8,129),
|
||||
"deepsets_output_dim": tune.lograndint(4,129),
|
||||
"head_hidden_n": tune.randint(1,6),
|
||||
"head_hidden_dim": tune.lograndint(16,257),
|
||||
"head_final_activation": tune.choice(['sigmoid', None]),
|
||||
}
|
||||
elif method == 'vd':
|
||||
ray_config = {
|
||||
"policy_lr": tune.loguniform(1e-5, 1e-3),
|
||||
"value_lr": tune.loguniform(1e-5, 1e-3),
|
||||
"policy_weight_decay": tune.choice([0, 0.1]),
|
||||
"value_weight_decay": tune.choice([0, 0.1]),
|
||||
"train_batch_size": tune.choice([16,32,64]),
|
||||
"deepsets_phi_hidden_n": tune.randint(1,5),
|
||||
"deepsets_phi_hidden_dim": tune.lograndint(8,65),
|
||||
"deepsets_latent_dim": tune.lograndint(8,129),
|
||||
"deepsets_rho_hidden_n": tune.randint(0,3),
|
||||
"deepsets_rho_hidden_dim": tune.lograndint(8,129),
|
||||
"deepsets_output_dim": tune.lograndint(4,129),
|
||||
"head_hidden_n": tune.randint(1,6),
|
||||
"head_hidden_dim": tune.lograndint(16,257),
|
||||
"head_final_activation": tune.choice(['sigmoid', None]),
|
||||
"clip_grad_norm": tune.choice([.5, 1., 5., 10.]),
|
||||
"discount": tune.choice([.95, .99])
|
||||
}
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return ray_config
|
||||
|
||||
if __name__ == '__main__':
|
||||
kwargs = parse_args()
|
||||
|
||||
# make prefix of output files
|
||||
|
||||
if kwargs['config_path']:
|
||||
# load config
|
||||
with open(kwargs['config_path'], 'r') as cfg:
|
||||
config = json5.load(cfg)
|
||||
if not os.path.isdir(kwargs['outdir']):
|
||||
os.makedirs(kwargs['outdir'])
|
||||
filestr = opj(kwargs['outdir'], basestr(**kwargs))
|
||||
if kwargs['ray']:
|
||||
filestr = kwargs['config_path'].replace('_config.json','')
|
||||
main(config, filestr=filestr, **kwargs)
|
||||
|
||||
elif kwargs['ray'] and kwargs['train']:
|
||||
|
||||
ray.shutdown()
|
||||
ray.init(log_to_driver=False)
|
||||
|
||||
def ray_train(config, datadir=None):
|
||||
full_config = get_full_config(config, kwargs['method'])
|
||||
main(full_config, filestr='exp', **kwargs)
|
||||
|
||||
ray_config = get_ray_config(kwargs['method'])
|
||||
search = HyperOptSearch(ray_config, max_concurrent=8, metric='cv_loss',mode="min",)
|
||||
custom_scheduler = ASHAScheduler(metric='cv_loss', mode="min", grace_period=15)
|
||||
|
||||
analysis = tune.run(
|
||||
ray_train,
|
||||
#config=ray_config,
|
||||
search_alg=search,
|
||||
scheduler=custom_scheduler,
|
||||
local_dir=kwargs['outdir'],
|
||||
#resources_per_trial={"cpu": 2},
|
||||
time_budget_s=120*60,
|
||||
num_samples=kwargs['nsamples'],
|
||||
)
|
||||
elif kwargs['ray'] and kwargs['test']:
|
||||
analysis = Analysis(kwargs['outdir'], default_metric="cv_loss", default_mode="min")
|
||||
config = analysis.get_best_config()
|
||||
filepath = analysis.get_best_logdir()
|
||||
filestr = opj(filepath, 'exp')
|
||||
config_path = filestr+'_config.json'
|
||||
with open(config_path, 'r') as cfg:
|
||||
config = json5.load(cfg)
|
||||
print("Best ray experiment:", filepath)
|
||||
main(config, filestr=filestr, **kwargs)
|
||||
else:
|
||||
raise Exception('No valid config found')
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
python experiments/experiment.py --ray --train -d ./expert_data/base
|
||||
python experiments/experiment.py --ray --test -d ./expert_data/base --nframes 1000
|
||||
python experiments/experiment.py --ray --train -d ./expert_data/reg
|
||||
python experiments/experiment.py --ray --test -d ./expert_data/reg --nframes 1000
|
||||
python experiments/experiment.py --ray --train -d ./expert_data/reg_graph --graph
|
||||
python experiments/experiment.py --ray --test -d ./expert_data/reg_graph --graph --nframes 1000
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
# python experiments/experiment.py --method vd --train --ray -d expert_data/reg -o output/vd/loc00/reg --nsamples 400
|
||||
# python experiments/experiment.py --test --ray --method vd -d expert_data/normal -o output/vd/loc00/normal --nframes 1000
|
||||
python experiments/experiment.py --train --method vd --config config/value_dice.json5
|
||||
@@ -53,6 +53,7 @@ def training_function(config):
|
||||
),
|
||||
check_collisions=True,
|
||||
stop_on_collision=config['trainenv']['stop_on_collision'],
|
||||
use_idm=config['trainenv']['use_idm'],
|
||||
), collision_distance=6, collision_penalty=100),
|
||||
lambda obs: (obs - obs_min) / (obs_max - obs_min + 1e-10)
|
||||
)) for _ in range(60)]
|
||||
@@ -67,6 +68,7 @@ def training_function(config):
|
||||
),
|
||||
check_collisions=True,
|
||||
stop_on_collision=config['trainenv']['stop_on_collision'],
|
||||
use_idm=config['trainenv']['use_idm'],
|
||||
track=track,
|
||||
), collision_distance=6, collision_penalty=100),
|
||||
lambda obs: (obs - obs_min) / (obs_max - obs_min + 1e-10)
|
||||
@@ -170,6 +172,7 @@ if __name__ == '__main__':
|
||||
'experiment': args.train,
|
||||
'trainenv': {
|
||||
'stop_on_collision': False,
|
||||
'use_idm': True,
|
||||
},
|
||||
'policy': {
|
||||
'learning_rate': 3e-4,
|
||||
@@ -236,3 +239,5 @@ if __name__ == '__main__':
|
||||
check_dir = analysis._checkpoints[i]['logdir']
|
||||
shutil.copyfile(os.path.join(check_dir,'policy_final.pt'),
|
||||
os.path.join(savepath, f'policy_seed{s}.pt'))
|
||||
shutil.copyfile(os.path.join(check_dir,'params.json'),
|
||||
os.path.join(savepath, 'config.json')) # copy config automatically
|
||||
@@ -1,10 +0,0 @@
|
||||
#DEFAULT PARAMETERS:
|
||||
# locs:list=None, (default to all locations)
|
||||
# tracks:list=None, (default to all tracks)
|
||||
# env_class:str='NRasterizedIncrementingAgent',
|
||||
# env_args:dict={width:36,height:36,m_per_px:2},
|
||||
# expert_class:str='NRasterizedRouteIncrementingAgent',
|
||||
# expert_args:dict={mu:0.001}):
|
||||
|
||||
# python -m src.data.expert --locs='[DR_USA_Roundabout_FT]' --tracks='[0]'
|
||||
python -m src.data.expert --locs='[DR_USA_Roundabout_FT]' --tracks='[0]'
|
||||
Binary file not shown.
65
intersimple-expert-rollout-setobs2.py
Normal file
65
intersimple-expert-rollout-setobs2.py
Normal file
@@ -0,0 +1,65 @@
|
||||
import torch
|
||||
import functools
|
||||
from src.core.sampling import rollout_sb3
|
||||
from intersim.envs import IntersimpleLidarFlatIncrementingAgent
|
||||
from intersim.envs.intersimple import speed_reward
|
||||
from intersim.expert import NormalizedIntersimpleExpert
|
||||
from src.util.wrappers import CollisionPenaltyWrapper, Setobs
|
||||
import numpy as np
|
||||
from gym.wrappers import TransformObservation
|
||||
|
||||
obs_min = np.array([
|
||||
[-1000, -1000, 0, -np.pi, -1e-1, 0.],
|
||||
[0, -np.pi, -20, -20, -np.pi, -1e-1],
|
||||
[0, -np.pi, -20, -20, -np.pi, -1e-1],
|
||||
[0, -np.pi, -20, -20, -np.pi, -1e-1],
|
||||
[0, -np.pi, -20, -20, -np.pi, -1e-1],
|
||||
[0, -np.pi, -20, -20, -np.pi, -1e-1],
|
||||
]).reshape(-1)
|
||||
|
||||
obs_max = np.array([
|
||||
[1000, 1000, 20, np.pi, 1e-1, 0.],
|
||||
[50, np.pi, 20, 20, np.pi, 1e-1],
|
||||
[50, np.pi, 20, 20, np.pi, 1e-1],
|
||||
[50, np.pi, 20, 20, np.pi, 1e-1],
|
||||
[50, np.pi, 20, 20, np.pi, 1e-1],
|
||||
[50, np.pi, 20, 20, np.pi, 1e-1],
|
||||
]).reshape(-1)
|
||||
|
||||
def main(track:int, loc:int=0):
|
||||
env = IntersimpleLidarFlatIncrementingAgent(
|
||||
loc=loc,
|
||||
track=track,
|
||||
n_rays=5,
|
||||
reward=functools.partial(
|
||||
speed_reward,
|
||||
collision_penalty=0
|
||||
),
|
||||
)
|
||||
|
||||
policy = NormalizedIntersimpleExpert(env, mu=0.001)
|
||||
|
||||
env = Setobs(TransformObservation(
|
||||
CollisionPenaltyWrapper(
|
||||
env,
|
||||
collision_distance=6, collision_penalty=100
|
||||
), lambda obs: (obs - obs_min) / (obs_max - obs_min + 1e-10)
|
||||
))
|
||||
print(env.nv, 'vehicles')
|
||||
expert_data = rollout_sb3(env, policy, n_episodes=150, max_steps_per_episode=200)
|
||||
|
||||
states, actions, rewards, dones = expert_data
|
||||
print(f'Expert mean episode length {(~dones).sum() / states.shape[0]}')
|
||||
print(f'Expert mean reward per episode {rewards[~dones].sum() / states.shape[0]}')
|
||||
print(f'Observation mean', states[~dones].mean(0))
|
||||
print(f'Observation std', states[~dones].std(0))
|
||||
|
||||
torch.save(expert_data, f'intersimple-expert-data-setobs2-loc{loc}-track{track}.pt')
|
||||
|
||||
def loop(tracks:list=[0]):
|
||||
for track in tracks:
|
||||
main(track)
|
||||
|
||||
if __name__=='__main__':
|
||||
import fire
|
||||
fire.Fire(loop)
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed1_tseed0_comparison.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed1_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed1_tseed0_summary.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed1_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed2_tseed0_comparison.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed2_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed2_tseed0_summary.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed2_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed3_tseed0_comparison.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed3_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed3_tseed0_summary.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed3_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed4_tseed0_comparison.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed4_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed4_tseed0_summary.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed4_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed5_tseed0_comparison.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed5_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expA/loc_r0t0/policy_seed5_tseed0_summary.pkl
Normal file
BIN
out/hail/expA/loc_r0t0/policy_seed5_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed1_tseed0_comparison.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed1_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed1_tseed0_summary.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed1_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed2_tseed0_comparison.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed2_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed2_tseed0_summary.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed2_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed3_tseed0_comparison.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed3_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed3_tseed0_summary.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed3_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed4_tseed0_comparison.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed4_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed4_tseed0_summary.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed4_tseed0_summary.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed5_tseed0_comparison.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed5_tseed0_comparison.pkl
Normal file
Binary file not shown.
BIN
out/hail/expB/loc_r0t4/policy_seed5_tseed0_summary.pkl
Normal file
BIN
out/hail/expB/loc_r0t4/policy_seed5_tseed0_summary.pkl
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user