{"cells":[{"cell_type":"markdown","source":["## Empirical modeling\n\nOur goal is to learn from data through construction of statistical models. Explicit in this is statistics which demands understanding and adherence to rules of probability. In this section, we will lay out the basic foundations of probability and give examples to assist in our understanding. Our general flow will be to understand the rules of probability and how to manipulate them, move to probability distributions and descriptive statistics, touch on likelihood and Bayes, and end with a discussion on inference. \n\n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"902dfd95-5093-43f3-a24e-ef255977893c"}}},{"cell_type":"markdown","source":["## Principals of Probability\n\nIn this section, we will lay out the principles guiding our models -- probability and statistics. \n\nOutline of discussion:\n \n* Stochastic processes -- chance regularities\n* Belief and probability\n* Rules of probability\n* Factoring joint probabilities\n\nGeneral references:\n \n+ Statistical Inference (9780534243128): Casella, George, Berger, Roger L. \n+ Probability Theory and Statistical Inference: Empirical Modeling with Observational Data (9781107185142): Spanos, A. \n+ Bayesian Models: A Statistical Primer for Ecologists (9780691159287): Hobbs, N. Thompson, Hooten, Mevin B. \n+ A First Course in Bayesian Statistical Methods (0387922997): Hoff, Peter D.\n\n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"09b2e8f4-87bf-4453-befa-034fa81b7b70"}}},{"cell_type":"markdown","source":["## Chance regularities\n\nThe processes we are interested in are non-deterministic (exhibits chance) where patterns reveal themselves (show regularities) after collection lots of data. Arguably, modern statistics started with quantifying betting odds in dice games such as the sum of spots (for example, see [Feldman and Morgan](http://jse.amstat.org/v11n2/feldman.html) for a discussion of HOG). To illustrate this, consider a game of dice where we throw 2 dice and sum the dots. Let's simulate this in python and look at the data."],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"abde7c1a-b4ff-4738-aab7-b128564ebe95"}}},{"cell_type":"code","source":["## simulate 500 rolls of 2 dice using random sampling with replacement\nimport random\nimport pandas as pd\nrandom.seed(13457)\ntotal_rolls = 1000 \nnumbers = range(0,total_rolls-1)\nevens = list(filter(lambda x: x % 2, numbers))\nodds = [x+1 for x in evens]\n\ndice = [1, 2, 3, 4, 5, 6]\nrolls = random.choices(dice,k=total_rolls)\nrolls1 = [rolls[i] for i in evens]\nrolls2 = [rolls[i] for i in odds]\n\ndf = pd.DataFrame({'die1' : rolls1,\n 'die2' : rolls2})\ndf.loc[:,'dots'] = df.sum(axis=1)\n\n## look at the first 10 dice throws\ndf.head(n=10)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"a7cd2ca0-eacf-4722-be8d-2ac1397133f9"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"datasetInfos":[],"data":"\n\n
\n \n \n | \n die1 | \n die2 | \n dots | \n
\n \n \n \n | 0 | \n 6 | \n 2 | \n 8 | \n
\n \n | 1 | \n 6 | \n 4 | \n 10 | \n
\n \n | 2 | \n 1 | \n 6 | \n 7 | \n
\n \n | 3 | \n 5 | \n 3 | \n 8 | \n
\n \n | 4 | \n 3 | \n 5 | \n 8 | \n
\n \n | 5 | \n 5 | \n 2 | \n 7 | \n
\n \n | 6 | \n 1 | \n 6 | \n 7 | \n
\n \n | 7 | \n 6 | \n 5 | \n 11 | \n
\n \n | 8 | \n 2 | \n 4 | \n 6 | \n
\n \n | 9 | \n 6 | \n 6 | \n 12 | \n
\n \n
\n
\n\n
\n \n \n | \n die1 | \n die2 | \n dots | \n
\n \n \n \n | 0 | \n 6 | \n 2 | \n 8 | \n
\n \n | 1 | \n 6 | \n 4 | \n 10 | \n
\n \n | 2 | \n 1 | \n 6 | \n 7 | \n
\n \n | 3 | \n 5 | \n 3 | \n 8 | \n
\n \n | 4 | \n 3 | \n 5 | \n 8 | \n
\n \n | 5 | \n 5 | \n 2 | \n 7 | \n
\n \n | 6 | \n 1 | \n 6 | \n 7 | \n
\n \n | 7 | \n 6 | \n 5 | \n 11 | \n
\n \n | 8 | \n 2 | \n 4 | \n 6 | \n
\n \n | 9 | \n 6 | \n 6 | \n 12 | \n
\n \n
\n
7 92\n6 79\n8 73\n9 47\n5 46\n10 40\n4 35\n3 32\n11 27\n12 15\n2 14\nName: dots, dtype: int64
","removedWidgets":[],"addedWidgets":{},"metadata":{},"type":"html","arguments":{}}},"output_type":"display_data","data":{"text/html":["\n7 92\n6 79\n8 73\n9 47\n5 46\n10 40\n4 35\n3 32\n11 27\n12 15\n2 14\nName: dots, dtype: int64
"]}}],"execution_count":0},{"cell_type":"code","source":["# plot the frequencies as counts\nh = df.hist(column='dots', bins=11)"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"633ff3f0-8db0-4989-bb7b-aec6b0d7b104"}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"datasetInfos":[],"data":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPyUlEQVR4nO3de2ydd33H8fdnDawXQy8rs0LKMIyqgJpxqcUKlZBDh9TRjlRsQKeCUtYt/3DpWKYtZZrYHwMiMRBIu6kC1mh0BAjdWlGNtQt4MAkqksIIJbBWEEpDSbi0pekqSrbv/vBTYRKnPjk+F//s90uKfM5zbt+fYr/z+LGfk1QVkqT2/MK4B5Ak9ceAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLhWjSTXJ/nLcc8hDYoBl46SpJI8a9xzSIsx4JLUKAOuFSvJC5LckeShJB8FTp532x8kuTvJj5LcnOSp3fbPdnf5rySHk7w2ydlJPpnkge7+n0vi147Gzk9CrUhJngj8C/CPwFnAx4Hf7m57GfAu4DXAWuDbwA6Aqnpp9xTPq6qJqvoosAW4F3gKMAm8DfA9KDR2Blwr1YXAE4D3VdVPq2on8MXutiuBD1XVHVX1E+Ba4MVJpo7zXD9lLvRP757rc+WbCGkZMOBaqZ4KHDgqtN+ed9tjl6mqw8APgXXHea53A3cDtyb5ZpKtQ5hXOmEGXCvVfcC6JJm37Ve6j98Fnv7YxiSnAb8EHFjoiarqoaraUlXPBH4L+KMkFw9nbKl3Blwr1eeBI8BbkqxJ8irgRd1t/wS8Icnzk/wi8E7g9qra391+EHjmY0+U5LIkz+r+Mfgx8L/dH2msDLhWpKp6FHgVcBVwP/Ba4Mbutl3AnwOfYG5P/VeBK+Y9/C+A7d1vnbwGOBf4d+Awc/8w/G1VzY5iHdLjiT+LkaQ2uQcuSY0y4JLUKAMuSY0y4JLUqDWjfLGzzz67pqam+nrsww8/zGmnnTbYgZY517w6uObVYSlr3rNnzw+q6ilHbx9pwKempti9e3dfj52dnWVmZmawAy1zrnl1cM2rw1LWnOTbC233EIokNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNWqkZ2JKozC19ZaRvdb+bZeO7LWko7kHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmN6ingSd6a5M4kX03ykSQnJzkryW1J7uo+njnsYSVJP7NowJOsA94CTFfV+cBJwBXAVmBXVZ0L7OquS5JGpNdDKGuAU5KsAU4FvgtsBLZ3t28HLh/4dJKk40pVLX6n5BrgHcAjwK1VdWWSB6rqjHn3ub+qjjmMkmQzsBlgcnLygh07dvQ16OHDh5mYmOjrsa1yzf3Ze+DBAU2zuPXrTl/yc/j3vDosZc0bNmzYU1XTR29fs9gDu2PbG4FnAA8AH0/yul5fuKquA64DmJ6erpmZmV4f+nNmZ2fp97Gtcs39uWrrLYMZpgf7r5xZ8nP497w6DGPNiwYc+A3gW1X1fYAkNwIvAQ4mWVtV9yVZCxwa6GRaMaZOIKhb1h8ZaYCllvVyDPwe4MIkpyYJcDGwD7gZ2NTdZxNw03BGlCQtZNE98Kq6PclO4A7gCPAl5g6JTAAfS3I1c5F/9TAHlST9vF4OoVBVbwfeftTmnzC3Ny5JGgPPxJSkRhlwSWqUAZekRhlwSWqUAZekRhlwSWqUAZekRhlwSWqUAZekRvV0JqakhZ3IG3Udz4m8gdf+bZcu+fW0crgHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1KieAp7kjCQ7k3w9yb4kL05yVpLbktzVfTxz2MNKkn6m1z3w9wOfqqpnA88D9gFbgV1VdS6wq7suSRqRRQOe5MnAS4EPAlTVo1X1ALAR2N7dbTtw+XBGlCQtJFX1+HdIng9cB3yNub3vPcA1wIGqOmPe/e6vqmMOoyTZDGwGmJycvGDHjh19DXr48GEmJib6emyrVsqa9x54sOf7Tp4CBx8Z4jDL0Imsef2604c7zIislM/tE7GUNW/YsGFPVU0fvb2XgE8DXwAuqqrbk7wf+DHw5l4CPt/09HTt3r27n/mZnZ1lZmamr8e2aqWseWrrLT3fd8v6I7xn75ohTrP8nMia92+7dMjTjMZK+dw+EUtZc5IFA97LMfB7gXur6vbu+k7ghcDBJGu7J18LHOprMklSXxYNeFV9D/hOkvO6TRczdzjlZmBTt20TcNNQJpQkLajX71XfDNyQ5InAN4E3MBf/jyW5GrgHePVwRpQkLaSngFfVl4Fjjr8wtzcuSRoDz8SUpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEatGfcAkno3tfWWkb7e/m2XjvT1dGLcA5ekRrkHvgqNei9O0nC4By5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktSongOe5KQkX0ryye76WUluS3JX9/HM4Y0pSTraieyBXwPsm3d9K7Crqs4FdnXXJUkj0lPAk5wDXAp8YN7mjcD27vJ24PKBTiZJelypqsXvlOwE3gU8CfjjqrosyQNVdca8+9xfVcccRkmyGdgMMDk5ecGOHTv6GvTw4cNMTEz09dhWDWvNew88OPDnHJTJU+DgI+OeYrRc85z1604fzzAjspSv5w0bNuypqumjty/6drJJLgMOVdWeJDMn+sJVdR1wHcD09HTNzJzwUwAwOztLv49t1bDWfNUyfjvZLeuP8J69q+tdjl3znP1XzoxnmBEZxtdzL581FwGvTPIK4GTgyUk+DBxMsraq7kuyFjg00MkkSY9r0WPgVXVtVZ1TVVPAFcCnq+p1wM3Apu5um4CbhjalJOkYS/k98G3Ay5PcBby8uy5JGpETOvBWVbPAbHf5h8DFgx9JktQLz8SUpEYZcElqlAGXpEYZcElqlAGXpEatrtO/lrGpBc6O3LL+yLI+a1LSeLkHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1Ch/jfA4Fvq1PklaTtwDl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapSn0ktaFkb99hX7t1060tcbBvfAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGrVowJM8LclnkuxLcmeSa7rtZyW5Lcld3cczhz+uJOkxveyBHwG2VNVzgAuBNyZ5LrAV2FVV5wK7uuuSpBFZNOBVdV9V3dFdfgjYB6wDNgLbu7ttBy4f0oySpAWkqnq/czIFfBY4H7inqs6Yd9v9VXXMYZQkm4HNAJOTkxfs2LGjr0EPHz7MxMREX4/tx94DD47stY5n8hQ4+Mi4pxgt17w6LIc1r193+khfbykN27Bhw56qmj56e88BTzIB/Afwjqq6MckDvQR8vunp6dq9e/eJTd6ZnZ1lZmamr8f2Y9RvLr+QLeuP8J69q+v/3HDNq8NyWPOo/0OHpTQsyYIB7+m3UJI8AfgEcENV3dhtPphkbXf7WuBQX5NJkvrSy2+hBPggsK+q3jvvppuBTd3lTcBNgx9PknQ8vXwPcxHwemBvki93294GbAM+luRq4B7g1UOZUJK0oEUDXlX/CeQ4N1882HEkSb3yTExJapQBl6RGGXBJapQBl6RGGXBJatTqOv1LkjqjPtv6+ktOG/hzugcuSY1qZg9874EHuWoZvD+JJC0X7oFLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1akkBT3JJkm8kuTvJ1kENJUlaXN8BT3IS8DfAbwLPBX43yXMHNZgk6fEtZQ/8RcDdVfXNqnoU2AFsHMxYkqTFpKr6e2DyO8AlVfX73fXXA79eVW866n6bgc3d1fOAb/Q569nAD/p8bKtc8+rgmleHpaz56VX1lKM3rlnCMFlg2zH/GlTVdcB1S3iduRdLdlfV9FKfpyWueXVwzavDMNa8lEMo9wJPm3f9HOC7SxtHktSrpQT8i8C5SZ6R5InAFcDNgxlLkrSYvg+hVNWRJG8C/g04CfhQVd05sMmOteTDMA1yzauDa14dBr7mvn+IKUkaL8/ElKRGGXBJatSyD3iSpyX5TJJ9Se5Mcs24ZxqFJCcl+VKST457llFIckaSnUm+3v1dv3jcMw1bkrd2n9NfTfKRJCePe6ZBS/KhJIeSfHXetrOS3Jbkru7jmeOccdCOs+Z3d5/bX0nyz0nOGMRrLfuAA0eALVX1HOBC4I2r5JT9a4B94x5ihN4PfKqqng08jxW+9iTrgLcA01V1PnO/CHDFeKcaiuuBS47athXYVVXnAru66yvJ9Ry75tuA86vq14D/Bq4dxAst+4BX1X1VdUd3+SHmvrDXjXeq4UpyDnAp8IFxzzIKSZ4MvBT4IEBVPVpVD4x1qNFYA5ySZA1wKivwPIqq+izwo6M2bwS2d5e3A5ePcqZhW2jNVXVrVR3prn6BufNmlmzZB3y+JFPAC4DbxzzKsL0P+BPg/8Y8x6g8E/g+8A/dYaMPJDlt3EMNU1UdAP4KuAe4D3iwqm4d71QjM1lV98HcDhrwy2OeZ9R+D/jXQTxRMwFPMgF8AvjDqvrxuOcZliSXAYeqas+4ZxmhNcALgb+rqhcAD7Pyvq3+Od1x343AM4CnAqcled14p9KwJfkz5g4L3zCI52si4EmewFy8b6iqG8c9z5BdBLwyyX7m3uHxZUk+PN6Rhu5e4N6qeuw7q53MBX0l+w3gW1X1/ar6KXAj8JIxzzQqB5OsBeg+HhrzPCORZBNwGXBlDegEnGUf8CRh7tjovqp677jnGbaquraqzqmqKeZ+qPXpqlrRe2ZV9T3gO0nO6zZdDHxtjCONwj3AhUlO7T7HL2aF/+B2npuBTd3lTcBNY5xlJJJcAvwp8Mqq+p9BPe+yDzhze6SvZ25P9Mvdn1eMeygN3JuBG5J8BXg+8M7xjjNc3XcbO4E7gL3MfS2uuNPLk3wE+DxwXpJ7k1wNbANenuQu4OXd9RXjOGv+a+BJwG1dw/5+IK/lqfSS1KYW9sAlSQsw4JLUKAMuSY0y4JLUKAMuSY0y4JLUKAMuSY36f0DgI/VpiP8YAAAAAElFTkSuQmCC\n","removedWidgets":[],"addedWidgets":{},"metadata":{},"type":"image","arguments":{}}},"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPyUlEQVR4nO3de2ydd33H8fdnDawXQy8rs0LKMIyqgJpxqcUKlZBDh9TRjlRsQKeCUtYt/3DpWKYtZZrYHwMiMRBIu6kC1mh0BAjdWlGNtQt4MAkqksIIJbBWEEpDSbi0pekqSrbv/vBTYRKnPjk+F//s90uKfM5zbt+fYr/z+LGfk1QVkqT2/MK4B5Ak9ceAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLhWjSTXJ/nLcc8hDYoBl46SpJI8a9xzSIsx4JLUKAOuFSvJC5LckeShJB8FTp532x8kuTvJj5LcnOSp3fbPdnf5rySHk7w2ydlJPpnkge7+n0vi147Gzk9CrUhJngj8C/CPwFnAx4Hf7m57GfAu4DXAWuDbwA6Aqnpp9xTPq6qJqvoosAW4F3gKMAm8DfA9KDR2Blwr1YXAE4D3VdVPq2on8MXutiuBD1XVHVX1E+Ba4MVJpo7zXD9lLvRP757rc+WbCGkZMOBaqZ4KHDgqtN+ed9tjl6mqw8APgXXHea53A3cDtyb5ZpKtQ5hXOmEGXCvVfcC6JJm37Ve6j98Fnv7YxiSnAb8EHFjoiarqoaraUlXPBH4L+KMkFw9nbKl3Blwr1eeBI8BbkqxJ8irgRd1t/wS8Icnzk/wi8E7g9qra391+EHjmY0+U5LIkz+r+Mfgx8L/dH2msDLhWpKp6FHgVcBVwP/Ba4Mbutl3AnwOfYG5P/VeBK+Y9/C+A7d1vnbwGOBf4d+Awc/8w/G1VzY5iHdLjiT+LkaQ2uQcuSY0y4JLUKAMuSY0y4JLUqDWjfLGzzz67pqam+nrsww8/zGmnnTbYgZY517w6uObVYSlr3rNnzw+q6ilHbx9pwKempti9e3dfj52dnWVmZmawAy1zrnl1cM2rw1LWnOTbC233EIokNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNWqkZ2JKozC19ZaRvdb+bZeO7LWko7kHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmN6ingSd6a5M4kX03ykSQnJzkryW1J7uo+njnsYSVJP7NowJOsA94CTFfV+cBJwBXAVmBXVZ0L7OquS5JGpNdDKGuAU5KsAU4FvgtsBLZ3t28HLh/4dJKk40pVLX6n5BrgHcAjwK1VdWWSB6rqjHn3ub+qjjmMkmQzsBlgcnLygh07dvQ16OHDh5mYmOjrsa1yzf3Ze+DBAU2zuPXrTl/yc/j3vDosZc0bNmzYU1XTR29fs9gDu2PbG4FnAA8AH0/yul5fuKquA64DmJ6erpmZmV4f+nNmZ2fp97Gtcs39uWrrLYMZpgf7r5xZ8nP497w6DGPNiwYc+A3gW1X1fYAkNwIvAQ4mWVtV9yVZCxwa6GRaMaZOIKhb1h8ZaYCllvVyDPwe4MIkpyYJcDGwD7gZ2NTdZxNw03BGlCQtZNE98Kq6PclO4A7gCPAl5g6JTAAfS3I1c5F/9TAHlST9vF4OoVBVbwfeftTmnzC3Ny5JGgPPxJSkRhlwSWqUAZekRhlwSWqUAZekRhlwSWqUAZekRhlwSWqUAZekRvV0JqakhZ3IG3Udz4m8gdf+bZcu+fW0crgHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1KieAp7kjCQ7k3w9yb4kL05yVpLbktzVfTxz2MNKkn6m1z3w9wOfqqpnA88D9gFbgV1VdS6wq7suSRqRRQOe5MnAS4EPAlTVo1X1ALAR2N7dbTtw+XBGlCQtJFX1+HdIng9cB3yNub3vPcA1wIGqOmPe/e6vqmMOoyTZDGwGmJycvGDHjh19DXr48GEmJib6emyrVsqa9x54sOf7Tp4CBx8Z4jDL0Imsef2604c7zIislM/tE7GUNW/YsGFPVU0fvb2XgE8DXwAuqqrbk7wf+DHw5l4CPt/09HTt3r27n/mZnZ1lZmamr8e2aqWseWrrLT3fd8v6I7xn75ohTrP8nMia92+7dMjTjMZK+dw+EUtZc5IFA97LMfB7gXur6vbu+k7ghcDBJGu7J18LHOprMklSXxYNeFV9D/hOkvO6TRczdzjlZmBTt20TcNNQJpQkLajX71XfDNyQ5InAN4E3MBf/jyW5GrgHePVwRpQkLaSngFfVl4Fjjr8wtzcuSRoDz8SUpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEatGfcAkno3tfWWkb7e/m2XjvT1dGLcA5ekRrkHvgqNei9O0nC4By5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktSongOe5KQkX0ryye76WUluS3JX9/HM4Y0pSTraieyBXwPsm3d9K7Crqs4FdnXXJUkj0lPAk5wDXAp8YN7mjcD27vJ24PKBTiZJelypqsXvlOwE3gU8CfjjqrosyQNVdca8+9xfVcccRkmyGdgMMDk5ecGOHTv6GvTw4cNMTEz09dhWDWvNew88OPDnHJTJU+DgI+OeYrRc85z1604fzzAjspSv5w0bNuypqumjty/6drJJLgMOVdWeJDMn+sJVdR1wHcD09HTNzJzwUwAwOztLv49t1bDWfNUyfjvZLeuP8J69q+tdjl3znP1XzoxnmBEZxtdzL581FwGvTPIK4GTgyUk+DBxMsraq7kuyFjg00MkkSY9r0WPgVXVtVZ1TVVPAFcCnq+p1wM3Apu5um4CbhjalJOkYS/k98G3Ay5PcBby8uy5JGpETOvBWVbPAbHf5h8DFgx9JktQLz8SUpEYZcElqlAGXpEYZcElqlAGXpEatrtO/lrGpBc6O3LL+yLI+a1LSeLkHLkmNMuCS1CgDLkmNMuCS1CgDLkmNMuCS1Ch/jfA4Fvq1PklaTtwDl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapSn0ktaFkb99hX7t1060tcbBvfAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGrVowJM8LclnkuxLcmeSa7rtZyW5Lcld3cczhz+uJOkxveyBHwG2VNVzgAuBNyZ5LrAV2FVV5wK7uuuSpBFZNOBVdV9V3dFdfgjYB6wDNgLbu7ttBy4f0oySpAWkqnq/czIFfBY4H7inqs6Yd9v9VXXMYZQkm4HNAJOTkxfs2LGjr0EPHz7MxMREX4/tx94DD47stY5n8hQ4+Mi4pxgt17w6LIc1r193+khfbykN27Bhw56qmj56e88BTzIB/Afwjqq6MckDvQR8vunp6dq9e/eJTd6ZnZ1lZmamr8f2Y9RvLr+QLeuP8J69q+v/3HDNq8NyWPOo/0OHpTQsyYIB7+m3UJI8AfgEcENV3dhtPphkbXf7WuBQX5NJkvrSy2+hBPggsK+q3jvvppuBTd3lTcBNgx9PknQ8vXwPcxHwemBvki93294GbAM+luRq4B7g1UOZUJK0oEUDXlX/CeQ4N1882HEkSb3yTExJapQBl6RGGXBJapQBl6RGGXBJatTqOv1LkjqjPtv6+ktOG/hzugcuSY1qZg9874EHuWoZvD+JJC0X7oFLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1akkBT3JJkm8kuTvJ1kENJUlaXN8BT3IS8DfAbwLPBX43yXMHNZgk6fEtZQ/8RcDdVfXNqnoU2AFsHMxYkqTFpKr6e2DyO8AlVfX73fXXA79eVW866n6bgc3d1fOAb/Q569nAD/p8bKtc8+rgmleHpaz56VX1lKM3rlnCMFlg2zH/GlTVdcB1S3iduRdLdlfV9FKfpyWueXVwzavDMNa8lEMo9wJPm3f9HOC7SxtHktSrpQT8i8C5SZ6R5InAFcDNgxlLkrSYvg+hVNWRJG8C/g04CfhQVd05sMmOteTDMA1yzauDa14dBr7mvn+IKUkaL8/ElKRGGXBJatSyD3iSpyX5TJJ9Se5Mcs24ZxqFJCcl+VKST457llFIckaSnUm+3v1dv3jcMw1bkrd2n9NfTfKRJCePe6ZBS/KhJIeSfHXetrOS3Jbkru7jmeOccdCOs+Z3d5/bX0nyz0nOGMRrLfuAA0eALVX1HOBC4I2r5JT9a4B94x5ihN4PfKqqng08jxW+9iTrgLcA01V1PnO/CHDFeKcaiuuBS47athXYVVXnAru66yvJ9Ry75tuA86vq14D/Bq4dxAst+4BX1X1VdUd3+SHmvrDXjXeq4UpyDnAp8IFxzzIKSZ4MvBT4IEBVPVpVD4x1qNFYA5ySZA1wKivwPIqq+izwo6M2bwS2d5e3A5ePcqZhW2jNVXVrVR3prn6BufNmlmzZB3y+JFPAC4DbxzzKsL0P+BPg/8Y8x6g8E/g+8A/dYaMPJDlt3EMNU1UdAP4KuAe4D3iwqm4d71QjM1lV98HcDhrwy2OeZ9R+D/jXQTxRMwFPMgF8AvjDqvrxuOcZliSXAYeqas+4ZxmhNcALgb+rqhcAD7Pyvq3+Od1x343AM4CnAqcled14p9KwJfkz5g4L3zCI52si4EmewFy8b6iqG8c9z5BdBLwyyX7m3uHxZUk+PN6Rhu5e4N6qeuw7q53MBX0l+w3gW1X1/ar6KXAj8JIxzzQqB5OsBeg+HhrzPCORZBNwGXBlDegEnGUf8CRh7tjovqp677jnGbaquraqzqmqKeZ+qPXpqlrRe2ZV9T3gO0nO6zZdDHxtjCONwj3AhUlO7T7HL2aF/+B2npuBTd3lTcBNY5xlJJJcAvwp8Mqq+p9BPe+yDzhze6SvZ25P9Mvdn1eMeygN3JuBG5J8BXg+8M7xjjNc3XcbO4E7gL3MfS2uuNPLk3wE+DxwXpJ7k1wNbANenuQu4OXd9RXjOGv+a+BJwG1dw/5+IK/lqfSS1KYW9sAlSQsw4JLUKAMuSY0y4JLUKAMuSY0y4JLUKAMuSY36f0DgI/VpiP8YAAAAAElFTkSuQmCC\n"}}],"execution_count":0},{"cell_type":"markdown","source":["In this simulated game, we notice:\n\n * we are not able to predice with certainty the next throw \n * some combinations occur with higher frequency \n * the shape of the tabulated data is roughly triangular"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"3ff45e45-5721-4874-98e7-b583b10270a9"}}},{"cell_type":"markdown","source":["## Belief and probability\n\nSo, from this and maybe experience, we see that 7's occur more frequently than 12's, 5's and 9's about the same, etc, so we have a beleif. How do we go from a 'hunch' to a quantified understanding of the game? The answer is statistics and probability.\n\n\n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"6fdbedcc-ba18-4ad6-9d96-56ffb1d98de0"}}},{"cell_type":"markdown","source":["## Sample space, outcomes and events\n\nTo get an understanding of probability, we need some definitions.\n\n1. outcomes -- mutually exclusive and exhaustive list of possible results in a model \n2. events -- sets containing zero or more outcomes, we define events we are interested in \n3. sample space -- set of all possible outcomes (must be collectively exhaustive and mutually exclusive). \n\nConsider the dice game.\n\nOutcomes: 2,3,4,5,6,7,8,9,10,11,12 \nEvents: up to us, what game we play, could be evens vs odds for instance \nSample space: 2,3,4,5,6,7,8,9,10,11,12 \n\nThrowing 3 coins, counting the heads, evens wins \n\nOutcomes: 0,1,2,3 \nEvents: {0,2}, {1,3} \nSample space: 0,1,2,3 \n\nTo conform with the set theory behind this, the event space (\\\\(\\mathcal{F}\\\\)) must conform to a series of conditions:\n\n1. the event space contains the sample space \\\\(S \\in \\mathcal{F}\\\\) \n2. the event space is closed under compliments, ie if \\\\(E \\in \\mathcal{F}, E^c \\in \\mathcal{F}\\\\) \n3. the event space is closed under countable unions, \\\\(E_i \\in \\mathcal{F}\\\\) for i=1,2,..., then \\\\((\\bigcup_{i=1}^{\\infty}E_i) \\in \\mathcal{F}\\\\)\n\nFor completeness, we would need to include the empty set and the sample space to both event lists if we had wanted to enumerate the events in the event space.\n\n\n\n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"a30c73bd-f021-4383-b913-a679d6478cf7"}}},{"cell_type":"markdown","source":["### Probability Space and Probability\n\nArmed with some definitions of outcomes and events, we are finally ready to talk about probability.\n\nA probability space is a combination of a sample space, event space, and probability function. The probability function is a real-valued function mapping events to the interval [0,1]. The probability function adheres to the Axioms of Probability (Kolmogorov Axioms). These are summarized as:\n \n1. the probability of an event is a real number on the interval [0,1] \n \n $$0 \\le P(E) \\le 1$$ \n
\n2. the probability of at least one event occurring is 1 \n \n $$P(S) = 1$$ where S is the sample space \n
\n3. countable mutually exclusive sets of events satisfy the following \n \n $$P(\\bigcup_{i=1}^\\infty E_i) = \\sum_{i=1}^\\infty P(E_i)$$ \n
\n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"cc55025e-aad4-457d-9c36-2719dafb601e"}}},{"cell_type":"markdown","source":["### GRADED EVALUATION (15 mins)\n\n1. You repeatedly roll a die (some large number of times, say 1M) and come up with the following probability of roll results for spots 1-6 in order: {0.14,0.10,0.20,0.14,0.14,0.28). The probability of rolling a 3 is greater than that of rolling a 4.\n\n a. True \n\n b. False \n \n \n2. The die in the previous question is fair. \n\n a. True \n\n b. False \n \n \n3. You toss a both a fair coin and a fair die simultaneously. The sample space is \n\n a. {(HT),(1,2,3,4,5,6)} \n\n b. {(H,1),(H,2),(H,3),(H,4),(H,5),(H,6),(T,1),(T,2),(T,3),(T,4),(T,5),(T,6)} \n \n \n4. In this game of tossing a coin and die, the die comes up 6. What is the probability of the coin landing heads?\n\n a. 0.5 \n\n b. 0.8 \n\n6. You do the above game twice, in the first trial, it came up (H,6). What is the probability that event will occur in the second go (0.00%)?\n\n a. 0.69\n\n b. 0.08 \n \n
\n
\n
"],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"2bf48d49-eed6-4805-a9a9-50a5f22c47e9"}}},{"cell_type":"code","source":[""],"metadata":{"application/vnd.databricks.v1+cell":{"title":"","showTitle":false,"inputWidgets":{},"nuid":"e01711d5-6591-43d6-829d-6ef785f53d23"}},"outputs":[],"execution_count":0}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"mimetype":"text/x-python","name":"python","pygments_lexer":"ipython3","codemirror_mode":{"name":"ipython","version":3},"version":"3.9.4","nbconvert_exporter":"python","file_extension":".py"},"application/vnd.databricks.v1+notebook":{"notebookName":"Fundamentals-lecture1-belief-and-probability-unGRADED (1)","dashboards":[],"notebookMetadata":{"pythonIndentUnit":2},"language":"python","widgets":{},"notebookOrigID":2372252794354875}},"nbformat":4,"nbformat_minor":0}