Popular garden plants that can cause poisoning if eaten or are likely to cause skin irritation and allergic reactions:

Acalypha irritant; poisonous
Acokanthera poisonous
Aconitum poisonous, irritant
Actaea poisonous, irritant to and via the skin
Aesculus poisonous
Aglaonema  poisonous, irritant
Agrostemma githago poisonous
Allamanda  poisonous; skin & eye irritant
Alocasia  poisonous; skin & eye irritant
Aloe poisonous
Alstroemeria  skin irritant
 Amaryllis belladona poisonous
Anthurium poisonous; skin & eye irritant
Apocynum  poisonous
Arisaema  poisonous; skin & eye irritant
Arum poisonous; skin & eye irritant
Asparagus may cause skin allergy, fruits poisonous
Atropa  poisonous; skin irritant
Brugmansia  poisonous
Brunfelsia poisonous
Caladium poisonous
Calla palustris  poisonous; skin & eye irritant
Chelidonium majus  poisonous; skin & eye irritant
Chrysanthemum skin irritant
Colchicum poisonous
Colocasia esculenta  poisonous; skin & eye irritant
Convallaria majalis  poisonous
Coriaria  poisonous
x Cuprocyparis leylandii  skin irritant
Daphne  poisonous; skin irritant
Datura  poisonous; skin irritant
Delphinium  poisonous
Dictamnus  skin irritant
Dieffenbachia  poisonous; skin & eye irritant
Digitalis  poisonous
Dracunculus  poisonous; skin & eye irritant
Echium skin irritant
Epipremnum  poisonous; skin & eye irritant
Euonymus somewhat poisonous
Euphorbia poisonous; skin & eye irritant,
Ficus benjamina  can cause skin allergy
Ficus carica  skin & eye irritant 
Fremontodendron - skin & eye irritant
Gaultheria  poisonous
Gelsemium  poisonous
Gloriosa superba  poisonous
Hedera  somewhat poisonous; skin irritant
Helleborus  poisonous, skin irritant
Heracleum mantegazzianum  severe skin irritant in sunlight
Hippeastrum (Amaryllis)  poisonous
Homeria  poisonous
Hyacinthoides (Bluebell)  poisonous
Hyacinthus  skin irritant
Hyoscyamus  poisonous
Hypericum perforatum  poisonous
Ipomoea  poisonous
Iris  poisonous; skin irritant
Kalmia  poisonous
Laburnocytisus ‘Adamii’  poisonous
Laburnum poisonous
Lagenaria poisonous
Lantana  poisonous; skin irritant
Ligustrum (privet)  poisonous
Lobelia (except bedding)  poisonous, skin & eye irritant  
Lupinus  poisonous
Lysichiton  poisonous; skin & eye irritant
Mandragora  poisonous
Mirabilis poisonous; skin irritant
Monstera deliciosa  poisonous; skin & eye irritant
Narcissus  poisonous; skin irritant
Nerium oleander poisonous
Nicotiana  poisonous
Oenanthe  poisonous
Opuntia microdasys  skin irritant
Ornithogalum poisonous; skin irritant
Papaver somniferum poisonous
Passiflora caerulea  poisonous
Pedilanthus  skin & eye irritant; poisonous
Philodendron poisonous; skin & eye irritant
Phytolacca  poisonous; skin irritant
Podophyllum  poisonous
Polygonatum  poisonous
Polyscias  poisonous; skin irritant
Primula obconica  skin irritant
Prunus laurocerasus poisonous
Prunus lusitanica  seed kernels are poisonous
Rhamnus  poisonous; skin irritant
Rhaphidophora poisonous; skin & eye irritant
Rhus verniciflua,R. radicans, R. succedanea, R. diversiloba,R. rydbergii,R. striata, R. toxicarium, R. vernix   poisonous; severe skin irritant
Ricinus communis  poisonous
Robinia pseudoacacia  poisonous
Ruta  severe skin irritant in bright sunlight
Sambucus  poisonous
Schefflera  skin irritant
Scilla  poisonous
Scopolia  poisonous  
Solandra  poisonous
Solanum  poisonous
Solanum pseudocapsicum  poisonous
Sophora  poisonous
Spartium junceum  poisonous
Spathiphyllum  poisonous; skin & eye irritant
Symphoricarpos  poisonous
Symphytum  poisonous
Syngonium  poisonous; skin & eye irritant
Tabernaemontana poisonous
Taxus poisonous
Thevetia  poisonous; skin irritant
Tulipa skin irritant
Veratrum  poisonous
Vitex  skin irritant
Wisteria  poisonous
Xanthosoma  poisonous; skin & eye irritant
 Zantedeschia  poisonous; skin & irritant eye