convert jupyter notebooks to sphynx tutorial (#155)

* convert jupyter notebooks to sphynx tutorial

* update download links

* habitat-sim demo, update image links

* add documentation link, remove PRs welcome

* nit

* remove habitat-api_structure.png

README.md

 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/facebookresearch/habitat-api/blob/master/LICENSE)
 [![CircleCI](https://circleci.com/gh/facebookresearch/habitat-api.svg?style=shield)](https://circleci.com/gh/facebookresearch/habitat-api)
-[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/facebookresearch/habitat-api/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
 
 Habitat-API
 ==============================
@@ -8,7 +7,7 @@ Habitat-API
 Habitat-API is a modular high-level library for end-to-end development in embodied AI --
 defining embodied AI tasks (e.g. navigation, instruction following, question answering), configuring embodied agents (physical form, sensors, capabilities), training these agents (via imitation or reinforcement learning, or no learning at all as in classical SLAM), and benchmarking their performance on the defined tasks using standard metrics.
 
-Habitat-API currently uses [`Habitat-Sim`](https://github.com/facebookresearch/habitat-sim) as the core simulator, but is designed with a modular abstraction for the simulator backend to maintain compatibility over multiple simulators.
+Habitat-API currently uses [`Habitat-Sim`](https://github.com/facebookresearch/habitat-sim) as the core simulator, but is designed with a modular abstraction for the simulator backend to maintain compatibility over multiple simulators. For documentation refer [here](https://aihabitat.org/habitat-api/).
 
 <p align="center">
   <img src="res/img/habitat_compressed.gif"  height="400">
@@ -48,14 +47,14 @@ If you use the Habitat platform in your research, please cite the following [tec
 ## Installation
 
 1. Clone the github repository and install habitat-api using the commands below. Note that python>=3.6 is required for working with habitat-api. All the development and testing was done using python3.6. Please use 3.6 to avoid possible issues.
-  
+
     ```bash
     cd habitat-api
     pip install -e .
     ```
-  
+
     The command above will install only habitat core API. To include habitat_baselines along with all additional requirements, use the command below instead:
-  
+
     ```bash
     cd habitat-api
     pip install -r requirements.txt
@@ -119,7 +118,7 @@ An important objective of Habitat-API is to make it easy for users to set up a v
 * `Episode`: a class for episode specification that includes the initial position and orientation of an Agent, a scene id, a goal position and optionally shortest paths to the goal. An episode is a description of one task instance for the agent.
 
 <p align="center">
-  <img src='res/img/habitat-api_structure.png' alt="teaser results" width="100%"/>
+  <img src='res/img/habitat_api_structure.png' alt="teaser results" width="100%"/>
   <p align="center"><i>Architecture of Habitat-API</i></p>
 </p>
 
@@ -165,5 +164,5 @@ The Habitat project would not have been possible without the support and contrib
 Habitat-API is MIT licensed. See the LICENSE file for details.
 
 ## References
-1. [Habitat: A Platform for Embodied AI Research](https://arxiv.org/abs/1904.01201). Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, Dhruv Batra. Tech report, arXiv:1904.01201, 2019. 
+1. [Habitat: A Platform for Embodied AI Research](https://arxiv.org/abs/1904.01201). Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, Dhruv Batra. Tech report, arXiv:1904.01201, 2019.
 

docs/source/index.rst

@@ -15,6 +15,9 @@ A modular high-level library to train embodied AI agents across a variety of tas
    :caption: Tutorials
 
    tutorials/quickstart
+   tutorials/habitat-sim-demo
+   tutorials/habitat-api-demo
+   tutorials/view-transform-warp
 
 .. toctree::
    :maxdepth: 1

docs/source/tutorials/habitat-api-demo.md

+# Habitat-API Demo
+
+[[Download notebook](https://dl.fbaipublicfiles.com/habitat/notebooks/habitat-api-demo.ipynb)]
+
+```python
+import habitat
+
+import numpy as np
+import random
+
+%matplotlib inline
+import matplotlib.pyplot as plt
+```
+
+All the boilerplate code in the habitat-sim to set sensor config
+and agent config is abstracted out in the Habitat-API config system.
+Default config is at habitat/config/default.py.
+You can override defaults by specifying them in a separate file and pass it to the get_config function
+or defrost the config object, override parameters and freeze the config.
+
+```python
+config = habitat.get_config(config_paths='../configs/tasks/pointnav_mp3d.yaml')
+config.defrost()
+config.DATASET.DATA_PATH = '../data/datasets/pointnav/mp3d/v1/val/val.json.gz'
+config.DATASET.SCENES_DIR = '../data/scene_datasets/'
+config.freeze()
+
+env = habitat.Env(config=config)
+```
+
+out:
+```bash
+2019-06-06 16:11:35,200 initializing sim Sim-v0
+2019-06-06 16:11:46,171 initializing task Nav-v0
+```
+
+### scene semantic annotations
+
+```python
+def print_scene_recur(scene, limit_output=10):
+    count = 0
+    for level in scene.levels:
+        print(
+            f"Level id:{level.id}, center:{level.aabb.center},"
+            f" dims:{level.aabb.sizes}"
+        )
+        for region in level.regions:
+            print(
+                f"Region id:{region.id}, category:{region.category.name()},"
+                f" center:{region.aabb.center}, dims:{region.aabb.sizes}"
+            )
+            for obj in region.objects:
+                print(
+                    f"Object id:{obj.id}, category:{obj.category.name()},"
+                    f" center:{obj.aabb.center}, dims:{obj.aabb.sizes}"
+                )
+                count += 1
+                if count >= limit_output:
+                    return None
+
+# Print semantic annotation information (id, category, bounding box details)
+# for the current scene in a hierarchical fashion
+scene = env.sim.semantic_annotations()
+print_scene_recur(scene, limit_output=15)
+
+env.close()
+# Note: Since only one OpenGL is allowed per process,
+# you have to close the current env before instantiating a new one.
+```
+
+out:
+```bash
+
+Level id:0, center:[11.0210495  3.996935   3.3452997], dims:[ 43.0625    8.19569 -30.1122 ]
+Region id:0_0, category:rec/game, center:[16.61225    2.7802274 11.577564 ], dims:[10.364299   5.5838847 -4.14447  ]
+Object id:0_0_0, category:ceiling, center:[16.5905   4.54488 11.269  ], dims:[9.984315  4.0917997 2.1377602]
+Object id:0_0_1, category:wall, center:[16.5865     2.6818905 13.4147   ], dims:[9.69278   0.5280709 5.4398193]
+Object id:0_0_2, category:wall, center:[21.6013     1.7400599 11.3493   ], dims:[3.5423203  0.41668844 3.921341  ]
+Object id:0_0_3, category:door, center:[11.5374     1.2431393 10.386599 ], dims:[1.2573967  2.5311599  0.41445923]
+Object id:0_0_4, category:door, center:[20.6332     1.2136002 13.5958   ], dims:[0.15834427 2.4860601  1.1674671 ]
+Object id:0_0_5, category:wall, center:[16.5946    2.66614   9.331001], dims:[9.72554    0.23693037 5.3787804 ]
+Object id:0_0_6, category:window, center:[16.5822    2.852209 13.596898], dims:[1.5934639  0.16375065 1.2588081 ]
+Object id:0_0_7, category:beam, center:[16.6094    5.32839  11.348299], dims:[0.5116577  0.35226822 3.8936386 ]
+Object id:0_0_8, category:floor, center:[16.586       0.07907867 11.406     ], dims:[10.48608    4.3792195  0.2833004]
+Object id:0_0_9, category:lighting, center:[11.798      1.9214487 11.313999 ], dims:[0.25683594 0.5076561  0.15560722]
+Object id:0_0_10, category:wall, center:[11.57       1.7476702 11.3347   ], dims:[3.54352    0.41701245 3.9231815 ]
+Object id:0_0_11, category:misc, center:[16.5943   2.29591 11.4341 ], dims:[10.428299  4.48172   4.676901]
+Object id:0_0_12, category:door, center:[11.5234     1.2489185 12.228199 ], dims:[1.2521439  2.5423803  0.46386147]
+Object id:0_0_13, category:door, center:[16.5833     1.1790485 13.490699 ], dims:[5.45306   0.3474083 2.4161606]
+Object id:0_0_14, category:window, center:[21.6362     1.2518396 12.2613   ], dims:[1.1998444  2.5486398  0.37800598]
+```
+
+### actions and sensors
+
+```python
+from PIL import Image
+from habitat_sim.utils import d3_40_colors_rgb
+import numpy as np
+
+def display_sample(rgb_obs, semantic_obs, depth_obs):
+    rgb_img = Image.fromarray(rgb_obs, mode="RGB")
+
+    semantic_img = Image.new("P", (semantic_obs.shape[1], semantic_obs.shape[0]))
+    semantic_img.putpalette(d3_40_colors_rgb.flatten())
+    semantic_img.putdata((semantic_obs.flatten() % 40).astype(np.uint8))
+    semantic_img = semantic_img.convert("RGBA")
+
+    depth_img = Image.fromarray((depth_obs * 255).astype(np.uint8), mode="L")
+
+    arr = [rgb_img, semantic_img, depth_img]
+
+    titles = ['rgb', 'semantic', 'depth']
+    plt.figure(figsize=(12 ,8))
+    for i, data in enumerate(arr):
+        ax = plt.subplot(1, 3, i+1)
+        ax.axis('off')
+        ax.set_title(titles[i])
+        plt.imshow(data)
+    plt.show()
+```
+
+```python
+config = habitat.get_config(config_paths='../configs/tasks/pointnav_mp3d.yaml')
+config.defrost()
+config.DATASET.DATA_PATH = '../data/datasets/pointnav/mp3d/v1/val/val.json.gz'
+config.DATASET.SCENES_DIR = '../data/scene_datasets/'
+config.SIMULATOR.AGENT_0.SENSORS = ['RGB_SENSOR', 'DEPTH_SENSOR', 'SEMANTIC_SENSOR']
+config.SIMULATOR.SEMANTIC_SENSOR.WIDTH = 256
+config.SIMULATOR.SEMANTIC_SENSOR.HEIGHT = 256
+config.SIMULATOR.TURN_ANGLE = 30
+config.freeze()
+
+env = habitat.Env(config=config)
+env.episodes = random.sample(env.episodes, 2)
+
+max_steps = 4
+
+action_mapping = {
+    0: 'stop',
+    1: 'move_forward',
+    2: 'turn left',
+    3: 'turn right'
+}
+
+for i in range(len(env.episodes)):
+    observations = env.reset()
+
+    display_sample(observations['rgb'], observations['semantic'], np.squeeze(observations['depth']))
+
+    count_steps = 0
+    while count_steps < max_steps:
+        action = random.choice(list(action_mapping.keys()))
+        print(action_mapping[action])
+        observations = env.step(action)
+        display_sample(observations['rgb'], observations['semantic'], np.squeeze(observations['depth']))
+
+        count_steps += 1
+        if env.episode_over:
+            break
+
+env.close()
+```
+
+out:
+
+<p align="center">
+  <img src='https://dl.fbaipublicfiles.com/habitat/notebooks/images/habitat-api-demo.png' alt="actions and sensors" width="100%"/>
+  <p align="center"><i>actions and sensors</i></p>
+</p>
\ No newline at end of file

docs/source/tutorials/habitat-sim-demo.md

+# Habitat-Sim Demo
+
+[[Download notebook]]() TODO(akadian): insert link
+
+```python
+import habitat_sim
+
+import random
+%matplotlib inline
+import matplotlib.pyplot as plt
+
+import numpy as np
+
+test_scene = "../data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb"
+
+sim_settings = {
+    "width": 256,  # Spatial resolution of the observations
+    "height": 256,
+    "scene": test_scene,  # Scene path
+    "default_agent": 0,
+    "sensor_height": 1.5,  # Height of sensors in meters
+    "color_sensor": True,  # RGB sensor
+    "semantic_sensor": True,  # Semantic sensor
+    "depth_sensor": True,  # Depth sensor
+    "seed": 1,
+}
+```
+
+### simulator config
+
+```python
+def make_cfg(settings):
+    sim_cfg = habitat_sim.SimulatorConfiguration()
+    sim_cfg.gpu_device_id = 0
+    sim_cfg.scene.id = settings["scene"]
+
+    # Note: all sensors must have the same resolution
+    sensors = {
+        "color_sensor": {
+            "sensor_type": habitat_sim.SensorType.COLOR,
+            "resolution": [settings["height"], settings["width"]],
+            "position": [0.0, settings["sensor_height"], 0.0],
+        },
+        "depth_sensor": {
+            "sensor_type": habitat_sim.SensorType.DEPTH,
+            "resolution": [settings["height"], settings["width"]],
+            "position": [0.0, settings["sensor_height"], 0.0],
+        },
+        "semantic_sensor": {
+            "sensor_type": habitat_sim.SensorType.SEMANTIC,
+            "resolution": [settings["height"], settings["width"]],
+            "position": [0.0, settings["sensor_height"], 0.0],
+        },
+    }
+
+    sensor_specs = []
+    for sensor_uuid, sensor_params in sensors.items():
+        if settings[sensor_uuid]:
+            sensor_spec = habitat_sim.SensorSpec()
+            sensor_spec.uuid = sensor_uuid
+            sensor_spec.sensor_type = sensor_params["sensor_type"]
+            sensor_spec.resolution = sensor_params["resolution"]
+            sensor_spec.position = sensor_params["position"]
+
+            sensor_specs.append(sensor_spec)
+
+    # Here you can specify the amount of displacement in a forward action and the turn angle
+    agent_cfg = habitat_sim.agent.AgentConfiguration()
+    agent_cfg.sensor_specifications = sensor_specs
+    agent_cfg.action_space = {
+        "move_forward": habitat_sim.agent.ActionSpec(
+            "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
+        ),
+        "turn_left": habitat_sim.agent.ActionSpec(
+            "turn_left", habitat_sim.agent.ActuationSpec(amount=30.0)
+        ),
+        "turn_right": habitat_sim.agent.ActionSpec(
+            "turn_right", habitat_sim.agent.ActuationSpec(amount=30.0)
+        ),
+    }
+
+    return habitat_sim.Configuration(sim_cfg, [agent_cfg])
+
+cfg = make_cfg(sim_settings)
+sim = habitat_sim.Simulator(cfg)
+```
+
+### scene semantic annotations
+
+```python
+def print_scene_recur(scene, limit_output=10):
+    print(f"House has {len(scene.levels)} levels, {len(scene.regions)} regions and {len(scene.objects)} objects")
+    print(f"House center:{scene.aabb.center} dims:{scene.aabb.sizes}")
+
+    count = 0
+    for level in scene.levels:
+        print(
+            f"Level id:{level.id}, center:{level.aabb.center},"
+            f" dims:{level.aabb.sizes}"
+        )
+        for region in level.regions:
+            print(
+                f"Region id:{region.id}, category:{region.category.name()},"
+                f" center:{region.aabb.center}, dims:{region.aabb.sizes}"
+            )
+            for obj in region.objects:
+                print(
+                    f"Object id:{obj.id}, category:{obj.category.name()},"
+                    f" center:{obj.aabb.center}, dims:{obj.aabb.sizes}"
+                )
+                count += 1
+                if count >= limit_output:
+                    return None
+
+# Print semantic annotation information (id, category, bounding box details)
+# about levels, regions and objects in a hierarchical fashion
+scene = sim.semantic_scene
+print_scene_recur(scene)
+```
+
+out:
+```bash
+House has 1 levels, 10 regions and 187 objects
+House center:[-2.7928102  1.3372793 -1.5051247] dims:[17.57338    2.9023628 -8.8595495]
+Level id:0, center:[-3.157365   1.3372804 -1.5051247], dims:[16.69967    2.9023607 -8.8595495]
+Region id:0_0, category:bedroom, center:[-8.821845   1.259409  -2.6915383], dims:[ 4.1633096  2.5356617 -4.207343 ]
+Object id:0_0_0, category:wall, center:[-8.86568    1.2817702 -2.73879  ], dims:[2.58148 4.5891  4.59182]
+Object id:0_0_1, category:ceiling, center:[-8.91329  2.20326 -2.80575], dims:[4.4761996 4.46008   0.7124357]
+Object id:0_0_2, category:misc, center:[-8.69572    1.1633401 -4.2134695], dims:[2.5021195  0.61951023 2.34074   ]
+Object id:0_0_3, category:curtain, center:[-10.9129      1.0454602  -2.9228697], dims:[2.134861   0.49171448 3.8549194 ]
+Object id:0_0_4, category:void, center:[-8.06444    1.4491596 -1.7219999], dims:[0.8975539 1.5347222 0.6184306]
+Object id:0_0_5, category:bed, center:[-8.71032    0.6567161 -2.7839994], dims:[1.2672672 2.0257597 2.45652  ]
+Object id:0_0_6, category:void, center:[-6.79918  1.40336 -1.91666], dims:[0.08472061 0.8195841  0.28476596]
+Object id:0_0_7, category:tv_monitor, center:[-10.9803    1.01896  -1.43764], dims:[1.0417404 0.5545361 1.2688993]
+Object id:0_0_9, category:chest_of_drawers, center:[-9.89281     0.31491923 -3.5474799 ], dims:[0.47650528 0.63675606 0.57509613]
+Object id:0_0_10, category:cushion, center:[-9.2041     0.5827892 -3.71507  ], dims:[1.0096397  0.31469202 0.90284204]
+```
+
+```python
+random.seed(sim_settings["seed"])
+sim.seed(sim_settings["seed"])
+
+# Set agent state
+agent = sim.initialize_agent(sim_settings["default_agent"])
+agent_state = habitat_sim.AgentState()
+agent_state.position = np.array([0.0, 0.072447, 0.0])
+agent.set_state(agent_state)
+
+# Get agent state
+agent_state = agent.get_state()
+print("agent_state: position", agent_state.position, "rotation", agent_state.rotation)
+```
+
+out:
+```bash
+agent_state: position [0.       0.072447 0.      ] rotation quaternion(1, 0, 0, 0)
+```
+
+```python
+from PIL import Image
+from habitat_sim.utils import d3_40_colors_rgb
+
+def display_sample(rgb_obs, semantic_obs, depth_obs):
+    rgb_img = Image.fromarray(rgb_obs, mode="RGBA")
+
+    semantic_img = Image.new("P", (semantic_obs.shape[1], semantic_obs.shape[0]))
+    semantic_img.putpalette(d3_40_colors_rgb.flatten())
+    semantic_img.putdata((semantic_obs.flatten() % 40).astype(np.uint8))
+    semantic_img = semantic_img.convert("RGBA")
+
+    depth_img = Image.fromarray((depth_obs / 10 * 255).astype(np.uint8), mode="L")
+
+    arr = [rgb_img, semantic_img, depth_img]
+    titles = ['rgb', 'semantic', 'depth']
+    plt.figure(figsize=(12 ,8))
+    for i, data in enumerate(arr):
+        ax = plt.subplot(1, 3, i+1)
+        ax.axis('off')
+        ax.set_title(titles[i])
+        plt.imshow(data)
+    plt.show()
+```
+
+### random actions
+
+```python
+total_frames = 0
+action_names = list(
+    cfg.agents[
+        sim_settings["default_agent"]
+    ].action_space.keys()
+)
+
+max_frames = 5
+
+while total_frames < max_frames:
+    action = random.choice(action_names)
+    print("action", action)
+    observations = sim.step(action)
+    rgb = observations["color_sensor"]
+    semantic = observations["semantic_sensor"]
+    depth = observations["depth_sensor"]
+
+    display_sample(rgb, semantic, depth)
+
+    total_frames += 1
+```
+
+out:
+
+<p align="center">
+  <img src='https://dl.fbaipublicfiles.com/habitat/notebooks/images/habitat-sim-demo.png' alt="actions and sensors"
+  width="100%"/>
+  <p align="center"><i>actions and sensors</i></p>
+</p>
\ No newline at end of file

docs/source/tutorials/quickstart.md

-# Quickstart 
+# Quickstart
 
 In this quickstart we will briefly introduce the habitat stack using which we will setup the pointnav task and step around in the environment.
 
@@ -6,7 +6,7 @@ In this quickstart we will briefly introduce the habitat stack using which we wi
 
 Habitat is a platform for embodied AI research that consists of:
 1. **Habitat-Sim**: A flexible, high-performance 3D simulator with configurable agents, multiple sensors, and generic 3D dataset handling (with built-in support for [MatterPort3D](https://niessner.github.io/Matterport/), [Gibson](http://gibsonenv.stanford.edu/database/) and other datasets). [[github-repo](https://github.com/facebookresearch/habitat-sim)]
-    
+
 1. **Habitat-API**: A modular high-level library for end-to-end development in embodied AI -- defining embodied AI tasks (e.g. navigation, instruction following, question answering), configuring embodied agents (physical form, sensors, capabilities), training these agents (via imitation or reinforcement learning, or no learning at all as in classical SLAM), and benchmarking their performance on the defined tasks using standard metrics. [[github-repo](https://github.com/facebookresearch/habitat-api)]
 
 For installing Habitat-Sim and Habitat-API follow instructions [here](https://github.com/facebookresearch/habitat-api#installation).
@@ -89,7 +89,7 @@ Running the above code will initialize an agent inside an environment, you can m
 Below is a demo of what the example output will look like:
 
 <p align="center">
-  <img src="https://i.imgur.com/5EaQUnf.png"  height="400">
+  <img src="https://dl.fbaipublicfiles.com/habitat/notebooks/images/quickstart.png"  height="400">
 </p>
 
 For more examples refer to [Habitat-API examples](https://github.com/facebookresearch/habitat-sim/tree/master/examples) and [Habitat-Sim examples](https://github.com/facebookresearch/habitat-sim/tree/master/examples).

docs/source/tutorials/view-transform-warp.md

+# View, Transform and Warp
+
+[[Download Notebook](https://dl.fbaipublicfiles.com/habitat/notebooks/relative_camera_views_transform_and_warping_demo.ipynb)]
+
+Demonstrates how to extract camera parameters in the scene and how these camera parameters relate to the given views. We create 2 cameras and use RGB and Depth information to construct transformation from a view of camera 1 to camera 2 and validate that transformation comparing projected and original views.
+
+```python
+import os
+import numpy as np
+import quaternion
+
+import matplotlib.pyplot as plt
+%matplotlib inline
+
+import habitat
+
+import torch.nn.functional as F
+import torch
+from torchvision.transforms import ToTensor
+```
+
+```python
+# Set up the environment for testing
+config = habitat.get_config(config_paths="../configs/tasks/pointnav_rgbd.yaml")
+config.defrost()
+config.DATASET.DATA_PATH = '../data/datasets/pointnav/habitat-test-scenes/v1/val/val.json.gz'
+config.DATASET.SCENES_DIR = '../data/scene_datasets/'
+config.freeze()
+
+# Can also do directly in the config file
+config.defrost()
+config.SIMULATOR.DEPTH_SENSOR.NORMALIZE_DEPTH = False
+config.freeze()
+
+# Intrinsic parameters, assuming width matches height. Requires a simple refactor otherwise
+W = config.SIMULATOR.DEPTH_SENSOR.WIDTH
+H = config.SIMULATOR.DEPTH_SENSOR.HEIGHT
+
+assert(W == H)
+hfov = float(config.SIMULATOR.DEPTH_SENSOR.HFOV) * np.pi / 180.
+
+
+env = habitat.Env(config=config)
+
+
+obs = env.reset()
+initial_state = env._sim.get_agent_state(0)
+init_translation = initial_state.position
+init_rotation = initial_state.rotation
+```
+
+out:
+```bash
+2019-06-11 10:03:34,049 initializing sim Sim-v0
+I0611 10:03:34.056092 64715 simulator.py:78] Loaded navmesh ../data/scene_datasets/habitat-test-scenes/skokloster-castle.navmesh
+2019-06-11 10:03:35,053 initializing task Nav-v0
+```
+
+```python
+# Randomly permute the rotation
+
+def uniform_quat(original_angle):
+    original_euler = quaternion.as_euler_angles(original_angle)
+    euler_angles = np.array([(np.random.rand() - 0.5) * np.pi / 9. + original_euler[0],
+                             (np.random.rand() - 0.5) * np.pi / 9. + original_euler[1],
+                             (np.random.rand() - 0.5) * np.pi / 9. + original_euler[2]])
+    quaternions = quaternion.from_euler_angles(euler_angles)
+
+
+    return quaternions
+```
+
+```python
+# Generate two random, overlapping views
+
+depths = []
+rgbs = []
+cameras = []
+
+
+for i in range(0, 2):
+    rotation = uniform_quat(init_rotation)
+    translation = init_translation + np.random.rand(3,) * 0.5 - 0.25
+
+    obs = env._sim.get_observations_at(position=translation, rotation=rotation, keep_agent_at_new_pose=True)
+    depths += [obs["depth"][...,0]]
+    rgbs += [obs["rgb"]]
+
+    cameras += [env._sim.get_agent_state()]
+
+env.close()
+```
+
+```python
+# Intrinsic parameters, K
+
+K = np.array([
+    [1 / np.tan(hfov / 2.), 0., 0., 0.],
+    [0., 1 / np.tan(hfov / 2.), 0., 0.],
+    [0., 0.,  1, 0],
+    [0., 0., 0, 1]])
+
+# Now get an approximation for the true world coordinates -- see if they make sense
+# [-1, 1] for x and [1, -1] for y as array indexing is y-down while world is y-up
+xs, ys = np.meshgrid(np.linspace(-1,1,W), np.linspace(1,-1,W))
+depth = depths[0].reshape(1,W,W)
+xs = xs.reshape(1,W,W)
+ys = ys.reshape(1,W,W)
+
+# Unproject
+# negate depth as the camera looks along -Z
+xys = np.vstack((xs * depth , ys * depth, -depth, np.ones(depth.shape)))
+xys = xys.reshape(4, -1)
+xy_c0 = np.matmul(np.linalg.inv(K), xys)
+
+# Now load in the cameras, are in the format camera --> world
+# Camera 1:
+quaternion_0 = cameras[0].sensor_states['depth'].rotation
+translation_0 = cameras[0].sensor_states['depth'].position
+rotation_0 = quaternion.as_rotation_matrix(quaternion_0)
+T_world_camera0 = np.eye(4)
+T_world_camera0[0:3,0:3] = rotation_0
+T_world_camera0[0:3,3] = translation_0
+
+# Camera 2:
+translation_1 = cameras[1].sensor_states['depth'].position
+quaternion_1 = cameras[1].sensor_states['depth'].rotation
+rotation_1 = quaternion.as_rotation_matrix(quaternion_1)
+T_world_camera1 = np.eye(4)
+T_world_camera1[0:3,0:3] =  rotation_1
+T_world_camera1[0:3,3] = translation_1
+
+# Invert to get world --> camera
+T_camera1_world = np.linalg.inv(T_world_camera1)
+
+# Transformation matrix between views
+# Aka the position of camera0 in camera1's coordinate frame
+T_camera1_camera0 = np.matmul(T_camera1_world, T_world_camera0)
+
+# Finally transform actual points
+xy_c1 = np.matmul(T_camera1_camera0, xy_c0)
+xy_newimg = np.matmul(K, xy_c1)
+
+# Normalize by negative depth
+xys_newimg = xy_newimg[0:2,:] / -xy_newimg[2:3,:]
+# Flip back to y-down to match array indexing
+xys_newimg[1] *= -1
+```
+
+```python
+# And visualise this new transformation
+
+# Create sampler
+sampler = torch.Tensor(xys_newimg).view(2, W, W).permute(1,2,0).unsqueeze(0)
+
+# Create generated image
+img1_tensor = ToTensor()(rgbs[0]).unsqueeze(0)
+img2_tensor = ToTensor()(rgbs[1]).unsqueeze(0)
+img2_warped = F.grid_sample(img2_tensor, sampler)
+
+# Visualise
+plt.figure(figsize=(10,10))
+ax1 = plt.subplot(221)
+ax1.imshow(img1_tensor.squeeze().permute(1,2,0))
+ax1.set_title("View 1", fontsize='large')
+ax1.axis('off')
+ax1 = plt.subplot(222)
+ax1.imshow(img2_tensor.squeeze().permute(1,2,0))
+ax1.set_title("View 2", fontsize='large')
+ax1.axis('off')
+ax1 = plt.subplot(223)
+plt.imshow(img2_warped.squeeze().permute(1,2,0))
+ax1.set_title("View 2 warped into View 1 \n according to the estimated transformation", fontsize='large')
+ax1.axis('off')
+ax1 = plt.subplot(224)
+ax1.imshow(np.abs(img2_warped.squeeze().permute(1,2,0) - img1_tensor.squeeze().permute(1,2,0)))
+ax1.set_title("Difference between warped \n and ground truth images", fontsize='large')
+ax1.axis('off')
+```
+
+out:
+```bash
+(-0.5, 255.5, 255.5, -0.5)
+```
+
+<p align="center">
+  <img src='https://dl.fbaipublicfiles.com/habitat/notebooks/images/view-transform-warp.png' alt="view-transform-warp" width="100%"/>
+  <p align="center"><i>actions and sensors</i></p>
+</p>
\ No newline at end of file