Polygenic scores, which estimate an individual's genetic propensity for a disease or trait, have the potential to become part of genomic healthcare. Neural-network based deep-learning has emerged as a method of intense interest to model complex, nonlinear phenomena, which may be adapted to exploit gene-gene and gene-environment interactions to potentially improve polygenic scores. We fit neural-network models to both simulated and 28 real traits in the UK Biobank. To infer the amount of nonlinearity present in a phenotype, we also present a framework using neural-networks, which controls for the potential confounding effect of linkage disequilibrium. Although we found evidence for small amounts of nonlinear effects, neural-network models were outperformed by linear regression models for both genetic-only and genetic+environmental input scenarios. In this work, we find that the usefulness of neural-networks for generating polygenic scores may currently be limited and confounded by joint tagging effects due to linkage disequilibrium.