{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# XID+ prior model for blind catalogues\n", "\n", "![HELP LOGO](https://avatars1.githubusercontent.com/u/7880370?s=100&v=4>)\n", "\n", "\n", "The final processing stage requires:\n", "1. Create xid+ prior files for all fields\n", "2. Description on runnning xid+\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pylab\n", "import pymoc\n", "import xidplus\n", "\n", "import numpy as np\n", "%matplotlib inline\n", "from astropy.table import Table\n", "import os\n", "import seaborn as sns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook uses all the raw data from the Blind SPIRE catalogue to create XID+ prior object and relevant tiling scheme" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Read in XID+SPIRE catalogue" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Herschel-Stripe-82_SPIRE\n" ] } ], "source": [ "name_field= ['AKARI-SEP_SPIRE','EGS_SPIRE','ELAIS-S1_SPIRE','SA13_SPIRE','XMM-13hr_SPIRE',\\\n", " 'COSMOS_SPIRE','ELAIS-N2_SPIRE','HDF-N_SPIRE','SPIRE-NEP_SPIRE','xFLS_SPIRE','ELAIS-N1_SPIRE',\\\n", " 'XMM-LSS_SPIRE','Lockman-SWIRE_SPIRE','CDFS-SWIRE_SPIRE', 'GAMA-09_SPIRE','GAMA-12_SPIRE',\\\n", " 'GAMA-15_SPIRE','SSDF_SPIRE','Bootes_SPIRE','HATLAS-NGP_SPIRE','HATLAS-SGP_SPIRE','AKARI-NEP_SPIRE',\\\n", " 'Herschel-Stripe-82_SPIRE']\n", "which_field = -1\n", "name_field = name_field[which_field] # select a field, in this case SA13\n", "print(name_field)\n", "loc = 'dmu22_'+name_field[0:-6]+'/data/'\n", "XID_table=Table.read(loc+name_field+'_all.fits')" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Table length=10\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
RADecF_BLIND_MF_SPIRE_250FErr_BLIND_MF_SPIRE_250F_BLIND_MF_SPIRE_350FErr_BLIND_MF_SPIRE_350F_BLIND_MF_SPIRE_500FErr_BLIND_MF_SPIRE_500rPRA_pixDec_pixF_BLIND_pix_SPIREFErr_BLIND_pix_SPIREflag
float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64
32.714274946082092.1640837313495953336.1118494825010.0093132173219382330.0151882138028394220.0062235769442848210.016396518580392290.009807836178185610.21328406120459231.032.714791010263282.163543638776656506.661816495524760.0121172849050397370.0
32.712726738100572.165704021863587144.337304865480040.009225252843298970.03625174618671450.0062235769442848210.0327296788003275940.0115894895566876370.192283103504627961.032.711694481121572.1651800842948994433.900815378937750.0123948849858013710.0
32.714275684415472.17531253003193875.56384084823317255.9306734120603145e-050.034729386504846020.0102152756558780870.044651049932407380.0111303266823222990.112370241099755381.032.713243337349922.1731845388677355.88708190704280.0077780875511725110.0
32.7147911157426562.16514776015321684.647968557101790.0131607577181724440.0080205258348068560.0066281819773540350.018309896133250570.009807836178185610.20094367413022321.032.714791221222022.1667518781552597330.31308648978510.0065186434399120970.0
3.4706316845027025-6.34587002922133912.0896409237461950.01128049794481019-0.054904716246471370.0076083218077093540.02302408227401050.0105626313520968790.246852370693837111.03.471941649443192-6.3456388734022955289.43583723417550.0164706855003853950.0
3.470108519949651-6.35971163929001289.207996852544010.006382801787814390.008151291477235410.008155795055867665-0.0057957079552225350.0119891134653932610.36857777007832891.03.4719424096354103-6.358428920333534276.151258485079150.0154998156775270520.0
3.464867404989823-6.336654906314079170.234758585451890.009445391245873583-0.089498371586301990.0075067920306131260.0124052669853765460.0120134228969621290.156541746100838571.03.4656534004432853-6.337475473462242275.48313304946590.0108801461992719290.0
3.4617240465946626-6.3429652877201587.5552287166908110.0093211374980626640.055874061668614180.00153520518098559240.0130739918495681260.0126070258508503360.033918866849498091.03.4625100208636788-6.3437858496721375251.658561703957870.009883752203542410.0
3.4653922569619677-6.3515907776939861.7178825115554230.0058828964182951360.0193801514198867870.0076329053062334580.0205785536781691180.0117261259077222030.253600242733045931.03.4656541609550864-6.350265582263374232.052165606631430.0105906514837796080.0
349.99485034151843-1.515528419425651834.6795773984020950.00017902899008364131-0.0207840014611934170.0108382304788107140.0265234914956646060.0126424077141886710.060734493224898011.0349.9966065147324-1.5175263317054863212.413234078036650.020880517042392630.0
" ], "text/plain": [ "\n", " RA Dec ... FErr_BLIND_pix_SPIRE flag \n", " float64 float64 ... float64 float64\n", "------------------ ------------------- ... -------------------- -------\n", " 32.71427494608209 2.1640837313495953 ... 0.012117284905039737 0.0\n", " 32.71272673810057 2.165704021863587 ... 0.012394884985801371 0.0\n", " 32.71427568441547 2.1753125300319387 ... 0.007778087551172511 0.0\n", "32.714791115742656 2.165147760153216 ... 0.006518643439912097 0.0\n", "3.4706316845027025 -6.345870029221339 ... 0.016470685500385395 0.0\n", " 3.470108519949651 -6.359711639290012 ... 0.015499815677527052 0.0\n", " 3.464867404989823 -6.336654906314079 ... 0.010880146199271929 0.0\n", "3.4617240465946626 -6.342965287720158 ... 0.00988375220354241 0.0\n", "3.4653922569619677 -6.351590777693986 ... 0.010590651483779608 0.0\n", "349.99485034151843 -1.5155284194256518 ... 0.02088051704239263 0.0" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "XID_table[0:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The uncertianties become Gaussian by $\\sim 20 \\mathrm{\\mu Jy}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Read in Maps" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "loc = '../dmu19/dmu19_HELP-SPIRE-maps/data/'\n", "version='1.1'\n", "name = [name_field+'250_v'+version+'.fits',name_field+'350_v'+version+'.fits',name_field+'500_v'+version+'.fits']\n", "pswfits=loc+name[0]\n", "pmwfits=loc+name[1]\n", "plwfits=loc+name[2]\n", "os.mkdir('./OUT_'+name_field[0:-6])\n", "output_folder ='./OUT_'+name_field[0:-6]+'/'\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "from astropy.io import fits\n", "from astropy import wcs\n", "\n", "#-----250-------------\n", "hdulist = fits.open(pswfits)\n", "im250phdu=hdulist[0].header\n", "im250hdu=hdulist[1].header\n", "\n", "im250=hdulist['IMAGE'].data*1.0E3 #convert to mJy\n", "nim250=hdulist['ERROR'].data*1.0E3 #convert to mJy\n", "w_250 = wcs.WCS(hdulist[1].header)\n", "pixsize250 = hdulist[\"Matchedfilter\"].header[\"PIXSIZE\"] \n", "\n", "hdulist.close()\n", "#-----350-------------\n", "hdulist = fits.open(pmwfits)\n", "im350phdu=hdulist[0].header\n", "im350hdu=hdulist[1].header\n", "\n", "im350=hdulist['IMAGE'].data*1.0E3 #convert to mJy\n", "nim350=hdulist['ERROR'].data*1.0E3 #convert to mJy\n", "w_350 = wcs.WCS(hdulist[1].header)\n", "pixsize350 = hdulist[\"Matchedfilter\"].header[\"PIXSIZE\"] \n", "hdulist.close()\n", "#-----500-------------\n", "hdulist = fits.open(plwfits)\n", "im500phdu=hdulist[0].header\n", "im500hdu=hdulist[1].header\n", "im500=hdulist['IMAGE'].data*1.0E3 #convert to mJy\n", "nim500=hdulist['ERROR'].data*1.0E3 #convert to mJy\n", "w_500 = wcs.WCS(hdulist[1].header)\n", "pixsize500 = hdulist[\"Matchedfilter\"].header[\"PIXSIZE\"] \n", "\n", "hdulist.close()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "## Set XID+ prior class\n", "ID = np.arange(1,np.size(XID_table['RA'])+1)\n", "ID = ID.astype(str)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: AstropyDeprecationWarning: \n", "Private attributes \"_naxis1\" and \"_naxis2\" have been deprecated since v3.1.\n", "Instead use the \"pixel_shape\" property which returns a list of NAXISj keyword values.\n", " [astropy.wcs.wcs]\n", "WARNING: AstropyDeprecationWarning: \n", "Private attributes \"_naxis1\" and \"_naxis2\" have been deprecated since v3.1.\n", "Instead use the \"pixel_shape\" property which returns a list of NAXISj keyword values.\n", " [astropy.wcs.wcs]\n" ] }, { "ename": "KeyboardInterrupt", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m#---prior250--------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mprior250\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mxidplus\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mim250\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mnim250\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mim250phdu\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mim250hdu\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;31m#Initialise with map, uncertianty map, wcs info and primary header\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mprior250\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprior_cat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mXID_table\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'RA'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mXID_table\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'Dec'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mname_field\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;34m'_250_XID.fits'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mID\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mID\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;31m#Set input catalogue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0mprior250\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprior_bkg\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m5.0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;31m#Set prior on background (assumes Gaussian pdf with mu and sigma)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m#---prior350--------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/Google_Drive/WORK/XID_plus/xidplus/prior.py\u001b[0m in \u001b[0;36mprior_cat\u001b[0;34m(self, ra, dec, prior_cat_file, flux_lower, flux_upper, ID, moc, z_median, z_sig)\u001b[0m\n\u001b[1;32m 138\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmoc\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcat_moc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 139\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 140\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcut_down_prior\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 141\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 142\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mset_tile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmoc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/Google_Drive/WORK/XID_plus/xidplus/prior.py\u001b[0m in \u001b[0;36mcut_down_prior\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 46\u001b[0m \u001b[0mCuts\u001b[0m \u001b[0mdown\u001b[0m \u001b[0mprior\u001b[0m \u001b[0;32mclass\u001b[0m \u001b[0mvariables\u001b[0m \u001b[0mto\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mMOC\u001b[0m \u001b[0massigned\u001b[0m \u001b[0mto\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mprior\u001b[0m \u001b[0;32mclass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 47\u001b[0m \"\"\"\n\u001b[0;32m---> 48\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcut_down_map\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 49\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcut_down_cat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 50\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/Google_Drive/WORK/XID_plus/xidplus/prior.py\u001b[0m in \u001b[0;36mcut_down_map\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 10\u001b[0m \"\"\"\n\u001b[1;32m 11\u001b[0m \u001b[0mwcs_temp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwcs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mWCS\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimhdu\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 12\u001b[0;31m \u001b[0mra\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdec\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwcs_temp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwcs_pix2world\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msx_pix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msy_pix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 13\u001b[0m \u001b[0mind_map\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmoc_routines\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheck_in_moc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mra\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdec\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmoc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 14\u001b[0m \u001b[0;31m# now cut down and flatten maps (default is to use all pixels, running segment will change the values below to pixels within segment)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/anaconda3/envs/xidplus/lib/python3.6/site-packages/astropy/wcs/wcs.py\u001b[0m in \u001b[0;36mwcs_pix2world\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1369\u001b[0m return self._array_converter(\n\u001b[1;32m 1370\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mxy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwcs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mp2s\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'world'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1371\u001b[0;31m 'output', *args, **kwargs)\n\u001b[0m\u001b[1;32m 1372\u001b[0m wcs_pix2world.__doc__ = \"\"\"\n\u001b[1;32m 1373\u001b[0m \u001b[0mTransforms\u001b[0m \u001b[0mpixel\u001b[0m \u001b[0mcoordinates\u001b[0m \u001b[0mto\u001b[0m \u001b[0mworld\u001b[0m \u001b[0mcoordinates\u001b[0m \u001b[0mby\u001b[0m \u001b[0mdoing\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/anaconda3/envs/xidplus/lib/python3.6/site-packages/astropy/wcs/wcs.py\u001b[0m in \u001b[0;36m_array_converter\u001b[0;34m(self, func, sky, ra_dec_order, *args)\u001b[0m\n\u001b[1;32m 1284\u001b[0m \"a 1-D array for each axis, followed by an origin.\")\n\u001b[1;32m 1285\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1286\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_return_list_of_arrays\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maxes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morigin\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1287\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1288\u001b[0m raise TypeError(\n", "\u001b[0;32m~/anaconda3/envs/xidplus/lib/python3.6/site-packages/astropy/wcs/wcs.py\u001b[0m in \u001b[0;36m_return_list_of_arrays\u001b[0;34m(axes, origin)\u001b[0m\n\u001b[1;32m 1238\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mra_dec_order\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0msky\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'input'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1239\u001b[0m \u001b[0mxy\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_denormalize_sky\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1240\u001b[0;31m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morigin\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1241\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mra_dec_order\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0msky\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'output'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1242\u001b[0m \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_normalize_sky\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/anaconda3/envs/xidplus/lib/python3.6/site-packages/astropy/wcs/wcs.py\u001b[0m in \u001b[0;36m\u001b[0;34m(xy, o)\u001b[0m\n\u001b[1;32m 1368\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"No basic WCS settings were created.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1369\u001b[0m return self._array_converter(\n\u001b[0;32m-> 1370\u001b[0;31m \u001b[0;32mlambda\u001b[0m \u001b[0mxy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwcs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mp2s\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxy\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'world'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1371\u001b[0m 'output', *args, **kwargs)\n\u001b[1;32m 1372\u001b[0m wcs_pix2world.__doc__ = \"\"\"\n", "\u001b[0;31mKeyboardInterrupt\u001b[0m: " ] } ], "source": [ "#---prior250--------\n", "prior250=xidplus.prior(im250,nim250,im250phdu,im250hdu)#Initialise with map, uncertianty map, wcs info and primary header\n", "prior250.prior_cat(XID_table['RA'],XID_table['Dec'],name_field+'_250_XID.fits',ID=ID)#Set input catalogue\n", "prior250.prior_bkg(-5.0,5)#Set prior on background (assumes Gaussian pdf with mu and sigma)\n", "#---prior350--------\n", "prior350=xidplus.prior(im350,nim350,im350phdu,im350hdu)\n", "prior350.prior_cat(XID_table['RA'],XID_table['Dec'],name_field+'_350_XID.fits',ID=ID)#Set input catalogue\n", "prior350.prior_bkg(-5.0,5)\n", "\n", "#---prior500--------\n", "prior500=xidplus.prior(im500,nim500,im500phdu,im500hdu)\n", "prior500.prior_cat(XID_table['RA'],XID_table['Dec'],name_field+'_500_XID.fits',ID=ID)#Set input catalogue\n", "prior500.prior_bkg(-5.0,5)\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "#pixsize array (size of pixels in arcseconds)\n", "pixsize=np.array([pixsize250,pixsize350,pixsize500])\n", "#point response function for the three bands\n", "prfsize=np.array([18.15,25.15,36.3])\n", "#use Gaussian2DKernel to create prf (requires stddev rather than fwhm hence pfwhm/2.355)\n", "from astropy.convolution import Gaussian2DKernel\n", "\n", "##---------fit using Gaussian beam-----------------------\n", "prf250=Gaussian2DKernel(prfsize[0]/2.355,x_size=101,y_size=101)\n", "prf250.normalize(mode='peak')\n", "prf350=Gaussian2DKernel(prfsize[1]/2.355,x_size=101,y_size=101)\n", "prf350.normalize(mode='peak')\n", "prf500=Gaussian2DKernel(prfsize[2]/2.355,x_size=101,y_size=101)\n", "prf500.normalize(mode='peak')\n", "\n", "pind250=np.arange(0,101,1)*1.0/pixsize[0] #get 250 scale in terms of pixel scale of map\n", "pind350=np.arange(0,101,1)*1.0/pixsize[1] #get 350 scale in terms of pixel scale of map\n", "pind500=np.arange(0,101,1)*1.0/pixsize[2] #get 500 scale in terms of pixel scale of map\n", "\n", "prior250.set_prf(prf250.array,pind250,pind250)#requires psf as 2d grid, and x and y bins for grid (in pixel scale)\n", "prior350.set_prf(prf350.array,pind350,pind350)\n", "prior500.set_prf(prf500.array,pind500,pind500)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----- There are 5 tiles required for input catalogue and 2 large tiles\n" ] } ], "source": [ "import pickle\n", "#from moc, get healpix pixels at a given order\n", "from xidplus import moc_routines\n", "\n", "order=7 \n", "tiles=moc_routines.get_HEALPix_pixels(order,prior250.sra,prior250.sdec,unique=True)\n", "order_large=6\n", "tiles_large=moc_routines.get_HEALPix_pixels(order_large,prior250.sra,prior250.sdec,unique=True)\n", "print('----- There are '+str(len(tiles))+' tiles required for input catalogue and '+str(len(tiles_large))+' large tiles')\n", "#output_folder='./'\n", "outfile=output_folder+'Master_prior.pkl'\n", "\n", "with open(outfile, 'wb') as f:\n", " xidplus.io.pickle.dump({'priors':[prior250,prior350,prior500],'tiles':tiles,'order':order,'version':xidplus.io.git_version()},f)\n", " \n", "outfile=output_folder+'Tiles.pkl'\n", "with open(outfile, 'wb') as f:\n", " pickle.dump({'tiles':tiles,'order':order,'tiles_large':tiles_large,'order_large':order_large,'version':xidplus.io.git_version()},f)\n" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "./OUT_SA13/\n" ] } ], "source": [ "print(output_folder)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## The Master_prior.pkl and Tiles.pkl are used to run XID+\n", "# Running on Apollo\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "% mv XID\\_plus\\_hier.sh OUT\\_\"name_field\"\n", "\n", "% cd OUT_\"name_field\"\n", "\n", "% module load sge\n", "\n", "% qsub -t 1-$n_hier -q mps.q -jc mps.short XID_plus_hier.sh \n", "\n", "### n_hier is the number of large tiles\n", "\n", "% cd ..\n", "\n", "% qsub -t 1-$n_tiles -pe openmp 4 -l h_rt=6:00:00 -l m_mem_free=13G -q mps.q XID_plus_tile.sh\n", "### n_hier is the number of small tiles\n", "### Then combine the Bayesian maps into one:\n", "\n", "% python make_combined_map.py\n", "\n", "% cd output\n", "\n", "% ls *cat.fits > cat_files \n", "\n", "% module load starlink/hikianalia-64bit\n", "\n", "% stilts tcat ifmt=fits in=@cat_files out=dmu22_XID+SPIRE_\"name_field\"_BLIND.fits\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## The data products are:\n", "### dmu22XID+SPIRE\"name_field\"_BLIND.fits\n", "#### dmu22_XID+SPIRE\\_psw_\"name_field\"_Bayes_Pval\n", "#### dmu22_XID+SPIRE\\_pmw_\"name_field\"_Bayes_Pval\n", "#### dmu22_XID+SPIRE\\_plw_\"name_field\"_Bayes_Pval" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Final validation of the data can be found at: \n", "# http://hedam.lam.fr/HELP/dataproducts/dmu22/\n", "# or\n", "# https://github.com/H-E-L-P/dmu_products/tree/master/dmu22\n", "# dmu22_\"name\\_field\"/XID+BLIND_\"name_field\"_final_processing.ipynb" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "*This is a default HELP jupyter notebook *\n", "\n", " ![HELP LOGO](https://avatars1.githubusercontent.com/u/7880370?s=75&v=4)\n", "\n", "**Authors**: S. Duivenvoorden, P.D. Hurley\n", "\n", " \n", "For a full description of the database and how it is organised in to `dmu_products` please the top level [readme](../readme.md).\n", " \n", "The Herschel Extragalactic Legacy Project, ([HELP](http://herschel.sussex.ac.uk/)), is a [European Commission Research Executive Agency](https://ec.europa.eu/info/departments/research-executive-agency_en)\n", "funded project under the SP1-Cooperation, Collaborative project, Small or medium-scale focused research project, FP7-SPACE-2013-1 scheme, Grant Agreement\n", "Number 607254.\n", "\n", "[Acknowledgements](http://herschel.sussex.ac.uk/acknowledgements)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }