ungleich-public/gmba_django
6
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Compare commits

base: ungleich-public:master
Branches Tags
ungleich-public:master
ungleich-public:10884-dockerify
ungleich-public:fix_refresh_data
ungleich-public:10112/mountain-range-form-change
ungleich-public:2.1
ungleich-public:1.0
ungleich-public:1.12
ungleich-public:1.11
ungleich-public:1.10
ungleich-public:1.9
ungleich-public:1.8
..
compare: ungleich-public:1.10
Branches Tags
ungleich-public:master
ungleich-public:10884-dockerify
ungleich-public:fix_refresh_data
ungleich-public:10112/mountain-range-form-change
ungleich-public:2.1
ungleich-public:1.0
ungleich-public:1.12
ungleich-public:1.11
ungleich-public:1.10
ungleich-public:1.9
ungleich-public:1.8

No commits in common. "master" and "1.10" have entirely different histories.
master ... 1.10

25 changed files with 147 additions and 817 deletions
Show stats Expand all files Collapse all files

53
Changelog.md
View file

@ -1,58 +1,5 @@
# CHANGELOG.md # CHANGELOG.md
## 2.1 (2022-09-29)
- issue#11013: non-public entries appearing as public
## 2.0 (2022-09-19) Dockerify https://redmine.ungleich.ch/issues/10884
Steps:
- Create .env.prod and .env.prod.db files
#### .env.prod
```
DEBUG=0
SECRET_KEY=secret_key_here
DJANGO_SETTINGS_MODULE=gmba_django.settings.production
ALLOWED_HOSTS=localhost,127.0.0.1,[::1]
POSTGRES_USER=user
POSTGRES_PASSWORD=password
POSTGRES_DB=app
POSTGRES_HOST=db
POSTGRES_PORT=5432
```
Building containers
- docker-compose -f docker-compose.prod.yml down -v --remove-orphans
- docker-compose -f docker-compose.prod.yml up -d --build
Copy db and adust
- docker cp gd.sql gmba_django_db_1:/gd.sql
- docker-compose exec db psql --username=psql_username_name --dbname=psql_db_name < /gd.sql
Migrate / check
- docker-compose exec web python manage.py migrate --noinput
Static resources
- docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear
## 1.12 (2022-01-19)
Bugfixes:
- issue#10112: Mountain Range admin form reorganization
- issue#10149: Fix for organization name empty case
- issue#10147: Fix atlas showing up as default type for many Resources
- issue#10142: Person admin form enhancements
## 1.11 (2022-01-05)
Bugfixes:
- issue#10111
- 'Gmba v2 id:' is editable at the moment, it shouldn't be editable
Upgrades:
- issue#10113
- Django from 3.2.5 to 3.2.11 and then to 4.0.1
## 1.10 (2022-01-05) ## 1.10 (2022-01-05)
Enhancements: Enhancements:

85
Dockerfile.prod
View file

@ -1,85 +0,0 @@
###########
# BUILDER #
###########
# pull official base image
FROM python:3.9.6-alpine as builder
# set work directory
WORKDIR /usr/src/gmba_django
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update \
&& apk add postgresql-dev gcc python3-dev musl-dev
# lint
RUN pip install --upgrade pip
RUN pip install flake8==3.9.2
COPY . .
#RUN flake8 --ignore=E501,F401 .
# install dependencies
COPY ./requirements.txt .
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/gmba_django/wheels -r requirements.txt
#########
# FINAL #
#########
# pull official base image
FROM python:3.9.6-alpine
# create directory for the app user
RUN mkdir -p /home/app
# create the app user
RUN addgroup -S app && adduser -S app -G app
# create the appropriate directories
ENV HOME=/home/app
ENV APP_HOME=/home/app/gmba_django
RUN mkdir $APP_HOME
RUN mkdir $APP_HOME/static
RUN mkdir $APP_HOME/media
#ADD gmba_django /data/app
#WORKDIR /data/app/gmba_django
# install dependencies
RUN apk update && apk add libpq
COPY --from=builder /usr/src/gmba_django/wheels /wheels
COPY --from=builder /usr/src/gmba_django/requirements.txt .
RUN pip install --no-cache /wheels/*
# copy entrypoint.prod.sh
COPY ./entrypoint.prod.sh /entrypoint.prod.sh
COPY ./init.sh /init.sh
RUN sed -i 's/\r$//g' /entrypoint.prod.sh
RUN chmod +x /entrypoint.prod.sh
# copy project
#COPY . $APP_HOME
ADD . /home/app/gmba_django
WORKDIR /home/app/gmba_django
# chown all the files to the app user
RUN chown -R app:app $APP_HOME
# change to the app user
USER app
#RUN python manage.py collectstatic --noinput
# run entrypoint.prod.sh
ENTRYPOINT ["/entrypoint.prod.sh"]
EXPOSE 8000
CMD ["gunicorn", "--bind", "[::]:8000", "--chdir", "/home/app/gmba_django", "--workers", "3", "gmba_django.wsgi:application"]

240
app/admin.py
View file

@ -1,53 +1,8 @@
from re import A
from unicodedata import name
from django.contrib import admin from django.contrib import admin
from django.utils.html import format_html from django.utils.html import format_html
from .models import * from .models import *
from django.forms import TextInput from django.forms import TextInput
from django.conf import settings from django.conf import settings
from django.http import HttpResponse
import csv
from django import forms
from django.contrib.admin.widgets import AutocompleteSelect
from django.contrib import messages # import messages
class autocomplete_fields_form(forms.ModelForm):
"""This form overrides autocomplete
organization field for Person class
"""
def __init__(self, *args, **kwargs):
super(autocomplete_fields_form, self).__init__(*args, **kwargs)
# self.fields['organization'].label = 'My new label'
class Meta:
model = Person
fields = '__all__'
widgets = {
'organization': forms.Select(attrs={'style': 'width:750px'})
}
def download_csv(modeladmin, request, queryset):
name = queryset[0].__class__.__name__
response = HttpResponse()
response['Content-Disposition'] = "attachment; filename="+name+"_table.csv"
first_row = []
for i in queryset[0]._meta.__dict__.get("fields"):
temp = str(i).split('.')
first_row.append(temp[-1])
writer = csv.writer(response)
writer.writerow(first_row)
for dic in queryset.values():
temp_list = (list(dic.values()))
writer.writerow(temp_list)
return response
download_csv.short_description = 'export (.csv)'
class PeopleResourceInline(admin.TabularInline): class PeopleResourceInline(admin.TabularInline):
@ -68,12 +23,11 @@ class PersonAdmin(admin.ModelAdmin):
PeopleRangeInline PeopleRangeInline
] ]
readonly_fields = ['id'] readonly_fields = ['id']
search_fields = ['title', 'first_name', 'last_name', 'organisation', search_fields = ['title', 'first_name', 'last_name', 'organisation', 'position', 'country__short_name', 'contact_email']
'position', 'country__short_name', 'contact_email']
fieldsets = ( fieldsets = (
(None, { (None, {
'fields': ( 'fields': (
('id', 'profile_on_web', 'news_letter'), ('id'),
('mr_mrs', 'title','full_name', 'position'), ('mr_mrs', 'title','full_name', 'position'),
('first_name', 'last_name', 'search_name', 'status'), ('first_name', 'last_name', 'search_name', 'status'),
('contact_email', 'email_2', 'skype', 'professional_phone'), ('contact_email', 'email_2', 'skype', 'professional_phone'),
@ -101,8 +55,8 @@ class PersonAdmin(admin.ModelAdmin):
}), }),
('Others', { ('Others', {
'classes': ('collapse',), 'classes': ('collapse',),
'fields': ( 'fields': ('orcid', 'web_of_science', 'twitter', 'instagram', 'updated',
'orcid', 'web_of_science', 'twitter', 'instagram', 'updated', ('news_letter', 'profile_on_web'),
'entry_date', 'entry_date',
'country', 'country',
'gmba_function' 'gmba_function'
@ -110,95 +64,9 @@ class PersonAdmin(admin.ModelAdmin):
}), }),
) )
list_display = ['id', 'full_name', 'organization'] list_display = ['id', 'full_name', 'organization']
autocomplete_fields = ['organization']
#autocomplete_fields = ['organization']
form = autocomplete_fields_form # adjust width of autocomplete_fields organization
list_per_page = settings.ADMIN_LIST_PER_PAGE list_per_page = settings.ADMIN_LIST_PER_PAGE
ordering = ['full_name'] # list_display_links = ['id']
actions = [download_csv]
def get_field_queryset(self, db, db_field, request):
queryset = super().get_field_queryset(db, db_field, request)
if db_field.name == 'organization':
queryset = queryset.order_by(
'country', 'organisation_english', 'organisation_2', 'organisation_3', 'acronym')
return queryset
def get_actions(self, request):
actions = super().get_actions(request)
if 'delete_selected' in actions:
del actions['delete_selected']
return actions
def delete_model(self, request, obj):
self.delete_person(request, obj)
return super(PersonAdmin, self).delete_model(request, obj)
def delete_person(self, request, obj):
taxa_people = TaxaPeople.objects.filter(person=obj.id)
for i in taxa_people:
i.delete()
ranges_people = RangesPeople.objects.filter(person=obj.id)
for i in ranges_people:
i.delete()
resources_people = ResourcesPeople.objects.filter(person=obj.id)
for i in resources_people:
i.delete()
scales_people = ScalesPeople.objects.filter(person=obj.id)
for i in scales_people:
i.delete()
people_function = PeopleFunction.objects.filter(person=obj.id)
for i in people_function:
i.delete()
people_range = PeopleRange.objects.filter(person=obj.id)
for i in people_range:
i.delete()
people_resources = PeopleResource.objects.filter(person=obj.id)
for i in people_resources:
i.delete()
fields_people = FieldsPeople.objects.filter(person=obj.id)
for i in fields_people:
i.delete()
methods_people = MethodsPeople.objects.filter(person=obj.id)
for i in methods_people:
i.delete()
def delete_persons(self, request, obj):
for o in obj.all():
taxa_people = TaxaPeople.objects.filter(person=o.id)
for i in taxa_people:
i.delete()
ranges_people = RangesPeople.objects.filter(person=o.id)
for i in ranges_people:
i.delete()
resources_people = ResourcesPeople.objects.filter(person=o.id)
for i in resources_people:
i.delete()
scales_people = ScalesPeople.objects.filter(person=o.id)
for i in scales_people:
i.delete()
people_function = PeopleFunction.objects.filter(person=o.id)
for i in people_function:
i.delete()
people_range = PeopleRange.objects.filter(person=o.id)
for i in people_range:
i.delete()
people_resources = PeopleResource.objects.filter(person=o.id)
for i in people_resources:
i.delete()
fields_people = FieldsPeople.objects.filter(person=o.id)
for i in fields_people:
i.delete()
methods_people = MethodsPeople.objects.filter(person=o.id)
for i in methods_people:
i.delete()
o.delete()
messages.success(request, "Successfully deleted")
delete_persons.short_description = 'Delete Person & Related Fields'
actions = [download_csv, delete_persons]
class ResourceKeywordInline(admin.TabularInline): class ResourceKeywordInline(admin.TabularInline):
@ -262,52 +130,6 @@ class ResourceAdmin(admin.ModelAdmin):
) )
list_per_page = settings.ADMIN_LIST_PER_PAGE list_per_page = settings.ADMIN_LIST_PER_PAGE
def get_actions(self, request):
actions = super().get_actions(request)
if 'delete_selected' in actions:
del actions['delete_selected']
return actions
def delete_model(self, request, obj):
self.delete_resource(request, obj)
return super(ResourceAdmin, self).delete_model(request, obj)
def delete_resource(self, request, obj):
resource_keyword = ResourceKeyword.objects.filter(resource=obj.id)
for i in resource_keyword:
i.delete()
resource_range = ResourceRange.objects.filter(resource_title=obj.id)
for i in resource_range:
i.delete()
resource_people = ResourcesPeople.objects.filter(resource=obj.id)
for i in resource_people:
i.delete()
resource_people_resource = PeopleResource.objects.filter(
resource=obj.id)
for i in resource_people_resource:
i.delete()
def delete_resources(self, request, obj):
for o in obj.all():
resource_keyword = ResourceKeyword.objects.filter(resource=o.id)
for i in resource_keyword:
i.delete()
resource_range = ResourceRange.objects.filter(resource_title=o.id)
for i in resource_range:
i.delete()
resource_people = ResourcesPeople.objects.filter(resource=o.id)
for i in resource_people:
i.delete()
resource_people_resource = PeopleResource.objects.filter(
resource=o.id)
for i in resource_people_resource:
i.delete()
o.delete()
messages.success(request, "Successfully deleted")
delete_resources.short_description = 'Delete Resource & Related Fields'
actions = [download_csv, delete_resources]
class RangeNameTranslationInline(admin.TabularInline): class RangeNameTranslationInline(admin.TabularInline):
autocomplete_fields = ['language_translation'] autocomplete_fields = ['language_translation']
@ -323,22 +145,23 @@ class RangeCountryInline(admin.TabularInline):
class RangeAdmin(admin.ModelAdmin): class RangeAdmin(admin.ModelAdmin):
autocomplete_fields = ['mother_range'] autocomplete_fields = ['mother_range']
# inlines = [ inlines = [
# RangeNameTranslationInline, RangeNameTranslationInline,
# RangeCountryInline RangeCountryInline
# ] ]
readonly_fields = ('id', 'gmba_v2_id') readonly_fields = ('id',)
search_fields = ['name', 'range_name', 'range_name_ascii', search_fields = ['name', 'range_name', 'range_name_ascii',
'level', 'level', 'level_1', 'level_2', 'level_3', 'peak_name', 'comments', 'source', 'level', 'level', 'level_1', 'level_2', 'level_3', 'peak_name', 'comments', 'source',
'name_fr', 'name_de', 'name_es', 'name_pt', 'name_cn', 'name_ru', 'name_tr'] 'name_fr', 'name_de', 'name_es', 'name_pt', 'name_cn', 'name_ru', 'name_tr']
fieldsets = ( fieldsets = (
(None, { (None, {
'fields': ( 'fields': (
('gmba_v2_id', 'range_name', 'range_name_language'), ('gmba_v2_id', 'range_name', 'range_name_language', 'map_unit'),
('map_unit', 'feature', 'gmba_narrow', 'select_300'), ('range_name_map', 'select_300', 'checked'),
('range_name_map', 'range_name_ascii', 'GMBA_v1_id'), ('range_name_ascii', 'GMBA_v1_id'),
('latitude', 'longitude', 'area'), ('latitude', 'longitude'),
('mother_range', 'countries', 'wiki_data_id', 'wiki_data_url'), ('mother_range', 'wiki_data_id', 'area'),
('feature', 'range_alternate_id', 'gmba_narrow'),
'source', 'source',
'id', 'id',
) )
@ -350,20 +173,19 @@ class RangeAdmin(admin.ModelAdmin):
('name_ru', 'name_tr', 'name_cn') ('name_ru', 'name_tr', 'name_cn')
) )
}), }),
('Range Countries', {
'classes': ('collapse',),
'fields': ('countries',),
}),
('Comments', { ('Comments', {
'classes': ('collapse',), 'classes': ('collapse',),
'fields': ('comments',), 'fields': ('comments',),
}),
('Others', {
'fields': ('range_alternate_id', )
}) })
) )
list_display = ['range_name', 'mother_range', 'countries'] list_display = ['range_name', 'mother_range', 'countries']
ordering = ['range_name'] ordering = ['range_name']
list_per_page = settings.ADMIN_LIST_PER_PAGE list_per_page = settings.ADMIN_LIST_PER_PAGE
actions = [download_csv]
class PeopleOrganizationInline(admin.TabularInline): class PeopleOrganizationInline(admin.TabularInline):
model = Person model = Person
@ -376,17 +198,14 @@ class OrganizationAdmin(admin.ModelAdmin):
PeopleOrganizationInline, PeopleOrganizationInline,
] ]
readonly_fields = ('org_num1',) readonly_fields = ('org_num1',)
search_fields = ['organisation_search', 'org_alpha_search', search_fields = ['organisation_search', 'org_alpha_search', 'organisation_2', 'organisation_3', 'subject']
'organisation_2', 'organisation_3', 'subject', 'organisation_english'] list_display = ['organisation_english', 'organisation_2', 'country']
list_display = ['org_num1', 'organisation_english',
'organisation_2', 'country']
fieldsets = ( fieldsets = (
(None, { (None, {
'fields': ( 'fields': (
'org_num1', 'org_num1',
('acronym', 'category', 'subject'), ('acronym', 'category', 'subject'),
'country', 'organisation_english',
('organisation_english', 'tel', 'email', 'url',),
('organisation_2', 'organisation_3'), ('organisation_2', 'organisation_3'),
'organisation_original', 'organisation_original',
) )
@ -395,18 +214,19 @@ class OrganizationAdmin(admin.ModelAdmin):
'classes': ('collapse',), 'classes': ('collapse',),
'fields': ( 'fields': (
('street', 'postcode', 'city'), ('street', 'postcode', 'city'),
('po_box', 'lat_long'), ('po_box', 'country', 'lat_long'),
) )
}), }),
('Other details', { ('Other details', {
'classes': ('collapse',), 'classes': ('collapse',),
'fields': ( 'fields': (('tel', 'email'),
'url',
'tags', 'tags',
), ),
}) })
) )
ordering = [ 'organisation_english' ] ordering = [ 'organisation_english' ]
actions = [download_csv]
def org_url(self, instance): def org_url(self, instance):
return format_html('<a href="{0}" target="_blank">{1}</a>', return format_html('<a href="{0}" target="_blank">{1}</a>',
@ -499,8 +319,7 @@ class SpeciesAdmin(admin.ModelAdmin):
class TaxonRangeAdmin(admin.ModelAdmin): class TaxonRangeAdmin(admin.ModelAdmin):
model = TaxonRange model = TaxonRange
search_fields = ['range__range_name', 'taxon', search_fields = ['range__range_name', 'taxon', 'subrange_or_region', 'distribution', 'source', 'remarks']
'subrange_or_region', 'distribution', 'source', 'remarks']
list_display = ['range_name', 'taxon'] list_display = ['range_name', 'taxon']
def range_name(self, obj): def range_name(self, obj):
@ -574,7 +393,6 @@ admin.site.register(Keyword, KeywordAdmin)
# admin.site.register(Scale) # admin.site.register(Scale)
# admin.site.register(Taxon) # admin.site.register(Taxon)
# admin.site.register(Field, FieldAdmin) # admin.site.register(Field, FieldAdmin)
admin.site.disable_action('delete_selected')
# LU models # LU models

80
app/convert.py
View file

@ -71,8 +71,7 @@ def reindex_data():
# Data update routine # Data update routine
def refresh_data(filename, fmt=None, update_existing=False): def refresh_data(filename, fmt=None):
print("refresh_data")
count = 0 count = 0
rowcount = 0 rowcount = 0
if not isfile(filename): if not isfile(filename):
@ -102,40 +101,18 @@ def refresh_data(filename, fmt=None, update_existing=False):
return None return None
if fmt['dataformat'] is DataFormat.PERSON_DETAIL: if fmt['dataformat'] is DataFormat.PERSON_DETAIL:
print(row)
person, source_id = get_by_id(row['ID'], Person) person, source_id = get_by_id(row['ID'], Person)
if not person: if not person:
person = Person.objects.filter(first_name=row['First name'], last_name=row['Last name']).first() person = Person.objects.filter(first_name=row['First name'], last_name=row['Last name']).first()
if person:
print("Fetched from DB")
else:
print("Does not exist in DB")
if not person: if not person:
person = Person(first_name=row['First name'], last_name=row['Last name'], source_id=row['ID']) person = Person(first_name=row['First name'], last_name=row['Last name'], source_id=row['ID'])
person.source_id = source_id
person.title = row['Title']
print(row)
person.organisation = row['Organisation English']
print("Country = %s" % row['country'] if 'country' in row else '')
if 'country' not in row or row['country'] is None or row['country'].strip() == '':
row['country'] = 0
c = Country.objects.get(id=row['country'])
person.country = c
person.position = row['Position']
person.biography = row['Biography']
person.contact_email = row['e-mail 1']
person.personal_url = fix_url(row['URL'])
person.save()
print("Created")
# Update data fields # Update data fields
if update_existing:
person.source_id = source_id person.source_id = source_id
person.title = row['Title'] person.title = row['Title']
print(row)
person.organisation = row['Organisation English'] person.organisation = row['Organisation English']
print("Country = %s" % row['country'] if 'country' in row else '') print("Country = %s" % row['country'])
if 'country' not in row or row['country'] is None or row['country'].strip() == '': if row['country'] is None or row['country'].strip() == '':
row['country'] = 0 row['country'] = 0
c = Country.objects.get(id=row['country']) c = Country.objects.get(id=row['country'])
person.country = c person.country = c
@ -143,31 +120,23 @@ def refresh_data(filename, fmt=None, update_existing=False):
person.biography = row['Biography'] person.biography = row['Biography']
person.contact_email = row['e-mail 1'] person.contact_email = row['e-mail 1']
person.personal_url = fix_url(row['URL']) person.personal_url = fix_url(row['URL'])
person.save()
with transaction.atomic(): with transaction.atomic():
research_methods = add_linked(person, Method, row['Methods']) research_methods = add_linked(person, Method, row['Methods'])
methods_people = [MethodsPeople.objects.get_or_create(method_id=m.id, person_id=person.id)[0] for m in research_methods] methods_people = [MethodsPeople.objects.get_or_create(method_id=m.id, person_id=person.id) for m in research_methods]
person.methodspeople_set.set(methods_people)
research_scales = add_linked(person, Scale, row['Scale']) research_scales = add_linked(person, Scale, row['Scale'])
scales_people = [ScalesPeople.objects.get_or_create(scale_id=s.id, person_id=person.id)[0] for s in research_scales] scales_people = [ScalesPeople.objects.get_or_create(scale_id=s.id, person_id=person.id) for s in research_scales]
person.scalespeople_set.set(scales_people)
research_taxa = add_linked(person, Taxon, row['Taxa']) research_taxa = add_linked(person, Taxon, row['Taxa'])
taxa_people = [TaxaPeople.objects.get_or_create(taxon_id=t.id, person_id=person.id)[0] for t in research_taxa] taxa_people = [TaxaPeople.objects.get_or_create(taxon_id=t.id, person_id=person.id) for t in research_taxa]
person.taxapeople_set.set(taxa_people)
research_fields = add_linked(person, Field, row['Field of expertise']) research_fields = add_linked(person, Field, row['Field of expertise'])
fields_people = [FieldsPeople.objects.get_or_create(field_id=f.id, person_id=person.id)[0] for f in research_fields] fields_people = [FieldsPeople.objects.get_or_create(field_id=f.id, person_id=person.id) for f in research_fields]
person.fieldspeople_set.set(fields_people)
# research_ranges = add_linked(person, MountainRange, row['MountainRange'])
# ranges_people = [RangesPeople.objects.get_or_create(range_id=r.id, person_id=person.id)[0] for r in research_ranges]
# person.rangespeople_set.set(ranges_people)
person.index() person.index()
person.save() person.save()
count = count + 1 count = count + 1
elif fmt['dataformat'] is DataFormat.RESOURCE_DETAIL: elif fmt['dataformat'] is DataFormat.RESOURCE_DETAIL:
res, source_id = get_by_id(row['ID'], Resource) res, source_id = get_by_id(row['ID'], Resource)
if not res: if not res: res = Resource(source_id=source_id)
res = Resource(source_id=source_id)
res.title = row['Title'] res.title = row['Title']
res.citation = row['Citation'] res.citation = row['Citation']
res.url = fix_url(row['URL'].strip('#')) # remove weird #formatting# res.url = fix_url(row['URL'].strip('#')) # remove weird #formatting#
@ -186,11 +155,9 @@ def refresh_data(filename, fmt=None, update_existing=False):
elif fmt['dataformat'] is DataFormat.PERSON_RESOURCE: elif fmt['dataformat'] is DataFormat.PERSON_RESOURCE:
rzs, source_id = get_by_id(row['Resource'], Resource, first=False) rzs, source_id = get_by_id(row['Resource'], Resource, first=False)
if not rzs or not rzs.first(): if not rzs or not rzs.first(): continue
continue
ppl, source_id = get_by_id(row['Person'], Person, first=False) ppl, source_id = get_by_id(row['Person'], Person, first=False)
if not ppl or not ppl.first(): if not ppl or not ppl.first(): continue
continue
for person in ppl: for person in ppl:
person.resources = [] person.resources = []
for r in rzs: for r in rzs:
@ -200,30 +167,15 @@ def refresh_data(filename, fmt=None, update_existing=False):
elif fmt['dataformat'] is DataFormat.PERSON_RANGE: elif fmt['dataformat'] is DataFormat.PERSON_RANGE:
rzs, source_id = get_by_id(row['MountainRange'], MountainRange, first=False) rzs, source_id = get_by_id(row['MountainRange'], MountainRange, first=False)
print(" range=%s, source_id=%s" % (rzs, source_id))
if not rzs or not rzs.first(): if not rzs or not rzs.first():
print(" --- No rzs, continue")
continue continue
ppl, source_id = get_by_id(row['Person'], Person, first=False) ppl, source_id = get_by_id(row['Person'], Person, first=False)
print(" +++ ppl=%s, source_id=%s" % (ppl, source_id)) if not ppl or not ppl.first(): continue
if not ppl or not ppl.first(): for person in ppl:
print(" --- No ppl, continue") person.ranges = []
continue for r in rzs:
with transaction.atomic(): person.ranges.append(r)
person = ppl.first()
research_ranges = add_linked(person, MountainRange, row['MountainRange'])
ranges_people = [PeopleRange.objects.get_or_create(range_id=r.id, person_id=person.id)[0] for r in research_ranges]
person.peoplerange_set.set(ranges_people)
# for person in ppl:
# research_ranges = add_linked(person, MountainRange, row['MountainRange'])
# ranges_people = [RangesPeople.objects.get_or_create(range_id=r.id, person_id=person.id)[0] for r in research_ranges]
# person.rangespeople_set.set(ranges_people)
# ranges_people = RangesPeople
# for r in rzs:
# ranges_people.append(r)
# person.rangespeople_set.set(ranges_people)
person.save() person.save()
#print(" *** Saved %s => %s (%s)" % (person, ranges_people, len(ranges_people)))
count = count + 1 count = count + 1
elif fmt['extension'] == 'geojson': elif fmt['extension'] == 'geojson':

8
app/management/commands/import.py
View file

@ -356,7 +356,7 @@ class Command(BaseCommand):
for k, v in self.csv_files_models_dict.items(): for k, v in self.csv_files_models_dict.items():
if v.strip().lower() == model_name.strip().lower(): if v.strip().lower() == model_name.strip().lower():
csv_file_name = k csv_file_name = k
if csv_file_name == '': if k == '':
raise Exception('Could not find a csv file name for model %s' % model_name) raise Exception('Could not find a csv file name for model %s' % model_name)
if csv_folder_path.endswith('/'): if csv_folder_path.endswith('/'):
file_path = '%s%s' % (csv_folder_path, csv_file_name) file_path = '%s%s' % (csv_folder_path, csv_file_name)
@ -452,12 +452,6 @@ def handle_object_dict(object_dict, model_name, debug=False):
for i in ['checked']: for i in ['checked']:
if i in object_dict: if i in object_dict:
object_dict[i] = True if object_dict[i].lower().strip() == 'true' else False object_dict[i] = True if object_dict[i].lower().strip() == 'true' else False
for i in ['select_300']:
if i in object_dict:
object_dict[i] = True if object_dict[i].lower().strip() == 'x' else False
for i in ['gmba_narrow']:
if i in object_dict:
object_dict[i] = True if object_dict[i].lower().strip() == 'x' else False
# area field can't be empty # area field can't be empty
if 'area' in object_dict: if 'area' in object_dict:
if object_dict['area'] == '': if object_dict['area'] == '':

18
app/migrations/0052_alter_mountainrange_select_300.py
View file

@ -1,18 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-13 04:08
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0051_auto_20220104_1608'),
]
operations = [
migrations.AlterField(
model_name='mountainrange',
name='select_300',
field=models.BooleanField(blank=True, max_length=12, null=True),
),
]

18
app/migrations/0053_alter_mountainrange_select_300.py
View file

@ -1,18 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-13 04:15
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0052_alter_mountainrange_select_300'),
]
operations = [
migrations.AlterField(
model_name='mountainrange',
name='select_300',
field=models.BooleanField(default=False),
),
]

18
app/migrations/0054_alter_mountainrange_gmba_narrow.py
View file

@ -1,18 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-13 04:37
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0053_alter_mountainrange_select_300'),
]
operations = [
migrations.AlterField(
model_name='mountainrange',
name='gmba_narrow',
field=models.BooleanField(default=False, verbose_name='GMBA Standard'),
),
]

18
app/migrations/0055_alter_mountainrange_wiki_data_url.py
View file

@ -1,18 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-13 04:44
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0054_alter_mountainrange_gmba_narrow'),
]
operations = [
migrations.AlterField(
model_name='mountainrange',
name='wiki_data_url',
field=models.URLField(blank=True, null=True),
),
]

18
app/migrations/0056_alter_mountainrange_range_alternate_id.py
View file

@ -1,18 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-13 05:05
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0055_alter_mountainrange_wiki_data_url'),
]
operations = [
migrations.AlterField(
model_name='mountainrange',
name='range_alternate_id',
field=models.CharField(blank=True, max_length=128, null=True),
),
]

23
app/migrations/0057_alter_person_mr_mrs_alter_resource_type.py
View file

@ -1,23 +0,0 @@
# Generated by Django 4.0.1 on 2022-01-17 18:26
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app', '0056_alter_mountainrange_range_alternate_id'),
]
operations = [
migrations.AlterField(
model_name='person',
name='mr_mrs',
field=models.TextField(blank=True, choices=[('', ''), ('Ms', 'Ms'), ('Mr', 'Mr')], null=True),
),
migrations.AlterField(
model_name='resource',
name='type',
field=models.TextField(blank=True, choices=[('', ''), ('Atlas', 'Atlas'), ('Book', 'Book'), ('Book chapter', 'Book chapter'), ('Case study', 'Case study'), ('Conservation project', 'Conservation project'), ('Dataset', 'Dataset'), ('Development project', 'Development project'), ('Journal article', 'Journal article'), ('LTER', 'LTER'), ('Map', 'Map'), ('Network', 'Network'), ('PEGASuS', 'PEGASuS'), ('Presentation', 'Presentation'), ('Report', 'Report'), ('Research project', 'Research project'), ('Research Site', 'Research Site'), ('Thesis', 'Thesis'), ('Video', 'Video'), ('Website', 'Website'), ('Working Group', 'Working Group')], null=True),
),
]

178
app/models.py
View file

@ -31,8 +31,7 @@ class Field(models.Model):
class FieldsPeople(models.Model): class FieldsPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey( person = models.ForeignKey('Person', models.DO_NOTHING, blank=True, null=True)
'Person', models.DO_NOTHING, blank=True, null=True)
field = models.ForeignKey(Field, models.DO_NOTHING, blank=True, null=True) field = models.ForeignKey(Field, models.DO_NOTHING, blank=True, null=True)
class Meta: class Meta:
@ -54,10 +53,8 @@ class Method(models.Model):
class MethodsPeople(models.Model): class MethodsPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey( person = models.ForeignKey('Person', models.DO_NOTHING, blank=True, null=True)
'Person', models.DO_NOTHING, blank=True, null=True) method = models.ForeignKey(Method, models.DO_NOTHING, blank=True, null=True)
method = models.ForeignKey(
Method, models.DO_NOTHING, blank=True, null=True)
class Meta: class Meta:
db_table = 'methods_people' db_table = 'methods_people'
@ -95,14 +92,10 @@ class MountainRange(models.Model):
#range_name_map = models.CharField(blank=True, null=True, unique=True, max_length=128) #range_name_map = models.CharField(blank=True, null=True, unique=True, max_length=128)
range_name = models.CharField(blank=True, null=True, max_length=128) range_name = models.CharField(blank=True, null=True, max_length=128)
range_name_ascii = models.CharField(blank=True, null=True, max_length=128) range_name_ascii = models.CharField(blank=True, null=True, max_length=128)
range_name_language = models.ForeignKey( range_name_language = models.ForeignKey(Language, models.DO_NOTHING, blank=True, null=True, to_field='id')
Language, models.DO_NOTHING, blank=True, null=True, to_field='id') mother_range = models.ForeignKey("self", models.DO_NOTHING, blank=True, null=True, to_field='id')
mother_range = models.ForeignKey( feature = models.ForeignKey(RangeType, models.DO_NOTHING, blank=True, null=True, to_field='id')
"self", models.DO_NOTHING, blank=True, null=True, to_field='id') map_unit = models.TextField(blank=True, null=True, choices=MAP_UNIT_CHOICES)
feature = models.ForeignKey(
RangeType, models.DO_NOTHING, blank=True, null=True, to_field='id')
map_unit = models.TextField(
blank=True, null=True, choices=MAP_UNIT_CHOICES)
level = models.TextField(blank=True, null=True) level = models.TextField(blank=True, null=True)
level_text = models.TextField(blank=True, null=True) level_text = models.TextField(blank=True, null=True)
level_1 = models.TextField(blank=True, null=True) level_1 = models.TextField(blank=True, null=True)
@ -120,16 +113,14 @@ class MountainRange(models.Model):
comments = models.TextField(blank=True, null=True) comments = models.TextField(blank=True, null=True)
checked = models.BooleanField(default=False) checked = models.BooleanField(default=False)
source = models.TextField(blank=True, null=True) source = models.TextField(blank=True, null=True)
range_alternate_id = models.CharField( range_alternate_id = models.TextField(blank=True, null=True)
blank=True, null=True, max_length=128)
geologic_region = models.TextField(blank=True, null=True) geologic_region = models.TextField(blank=True, null=True)
gmba_v2_id = models.PositiveIntegerField(blank=True, null=True) gmba_v2_id = models.PositiveIntegerField(blank=True, null=True)
gmba_v2_id_str = models.TextField(blank=True, null=True) gmba_v2_id_str = models.TextField(blank=True, null=True)
wiki_data_id = models.CharField(blank=True, null=True, max_length=25) wiki_data_id = models.CharField(blank=True, null=True, max_length=25)
wiki_data_url = models.URLField(blank=True, null=True) wiki_data_url = models.TextField(blank=True, null=True)
select_300 = models.BooleanField(default=False) select_300 = models.CharField(blank=True, null=True, max_length=12)
gmba_narrow = models.BooleanField( gmba_narrow = models.CharField(blank=True, null=True, max_length=12)
default=False, verbose_name="GMBA Standard")
name_fr = models.CharField(blank=True, null=True, max_length=128) name_fr = models.CharField(blank=True, null=True, max_length=128)
name_de = models.CharField(blank=True, null=True, max_length=128) name_de = models.CharField(blank=True, null=True, max_length=128)
name_es = models.CharField(blank=True, null=True, max_length=128) name_es = models.CharField(blank=True, null=True, max_length=128)
@ -172,8 +163,7 @@ class Resource(models.Model):
("*****", "*****"), ("*****", "*****"),
) )
TYPE_CHOICES = ( TYPE_CHOICES = (
("", ""), ("", "Atlas"),
("Atlas", "Atlas"),
("Book", "Book"), ("Book", "Book"),
("Book chapter", "Book chapter"), ("Book chapter", "Book chapter"),
("Case study", "Case study"), ("Case study", "Case study"),
@ -210,8 +200,7 @@ class Resource(models.Model):
PEGASuS_Check_map_with_author = models.BooleanField(default=False) PEGASuS_Check_map_with_author = models.BooleanField(default=False)
PEGASuS_polygon_ID = models.CharField(max_length=32, blank=True, null=True) PEGASuS_polygon_ID = models.CharField(max_length=32, blank=True, null=True)
PEGASuS_Polygon_comments = models.TextField(blank=True, null=True) PEGASuS_Polygon_comments = models.TextField(blank=True, null=True)
PEGASuS_Assessment_ID = models.CharField( PEGASuS_Assessment_ID = models.CharField(max_length=32, blank=True, null=True)
max_length=32, blank=True, null=True)
gloria = models.BooleanField(default=False) gloria = models.BooleanField(default=False)
gnomo = models.BooleanField(default=False) gnomo = models.BooleanField(default=False)
lter = models.BooleanField(default=False) lter = models.BooleanField(default=False)
@ -362,20 +351,15 @@ class GMBA_function(models.Model):
class Organisation(models.Model): class Organisation(models.Model):
CATEGORY_CHOICES = ( CATEGORY_CHOICES = (
('Independent', 'Independent'), ('Independent', 'Independent'),
('Public administration > Research Institution', ('Public administration > Research Institution', 'Public administration > Research Institution'),
'Public administration > Research Institution'),
('Academia > University > Institute', 'Academia > University > Institute'), ('Academia > University > Institute', 'Academia > University > Institute'),
('Academia > University', 'Academia > University'), ('Academia > University', 'Academia > University'),
('Public administration > Other Agency', ('Public administration > Other Agency', 'Public administration > Other Agency'),
'Public administration > Other Agency'),
('Private > NGO (not for profit)', 'Private > NGO (not for profit)'), ('Private > NGO (not for profit)', 'Private > NGO (not for profit)'),
('Academia > Academy of Sciences', 'Academia > Academy of Sciences'), ('Academia > Academy of Sciences', 'Academia > Academy of Sciences'),
('Academia > Academy of Sciences > Institute', ('Academia > Academy of Sciences > Institute', 'Academia > Academy of Sciences > Institute'),
'Academia > Academy of Sciences > Institute'), ('Public administration > Government', 'Public administration > Government'),
('Public administration > Government', ('Intergovernmental Agency > Research Institution', 'Intergovernmental Agency > Research Institution'),
'Public administration > Government'),
('Intergovernmental Agency > Research Institution',
'Intergovernmental Agency > Research Institution'),
('Intergovernmental Agency', 'Intergovernmental Agency '), ('Intergovernmental Agency', 'Intergovernmental Agency '),
('Private > Research Institution', 'Private > Research Institution'), ('Private > Research Institution', 'Private > Research Institution'),
('Collection > Museum', 'Collection > Museum'), ('Collection > Museum', 'Collection > Museum'),
@ -399,8 +383,7 @@ class Organisation(models.Model):
('Culture / arts', 'Culture / arts'), ('Culture / arts', 'Culture / arts'),
('Geography', 'Geography'), ('Geography', 'Geography'),
('Zoology', 'Zoology'), ('Zoology', 'Zoology'),
('Development / poverty / human rights', ('Development / poverty / human rights', 'Development / poverty / human rights'),
'Development / poverty / human rights'),
('Social Sciences', 'Social Sciences'), ('Social Sciences', 'Social Sciences'),
('Earth Sciences', 'Earth Sciences'), ('Earth Sciences', 'Earth Sciences'),
('Health', 'Health'), ('Health', 'Health'),
@ -415,12 +398,9 @@ class Organisation(models.Model):
org_num1 = models.AutoField(primary_key=True, verbose_name='ID') org_num1 = models.AutoField(primary_key=True, verbose_name='ID')
organisation_search = models.TextField(blank=True, null=True) organisation_search = models.TextField(blank=True, null=True)
org_alpha_search = models.TextField(blank=True, null=True) org_alpha_search = models.TextField(blank=True, null=True)
organisation_english = models.CharField( organisation_english = models.CharField(max_length=256, blank=True, null=True, verbose_name='Organisation')
max_length=256, blank=True, null=True, verbose_name='Organisation') organisation_2 = models.CharField(max_length=256, blank=True, null=True, verbose_name='Department 1')
organisation_2 = models.CharField( organisation_3 = models.CharField(max_length=256, blank=True, null=True, verbose_name='Department 2')
max_length=256, blank=True, null=True, verbose_name='Department 1')
organisation_3 = models.CharField(
max_length=256, blank=True, null=True, verbose_name='Department 2')
organisation_original = models.TextField(blank=True, null=True) organisation_original = models.TextField(blank=True, null=True)
acronym = models.CharField(max_length=128, blank=True, null=True) acronym = models.CharField(max_length=128, blank=True, null=True)
street = models.CharField(max_length=256, blank=True, null=True) street = models.CharField(max_length=256, blank=True, null=True)
@ -433,33 +413,27 @@ class Organisation(models.Model):
url = models.URLField(blank=True, null=True) url = models.URLField(blank=True, null=True)
tel = models.CharField(max_length=128, blank=True, null=True) tel = models.CharField(max_length=128, blank=True, null=True)
email = models.EmailField(blank=True, null=True) email = models.EmailField(blank=True, null=True)
country = models.ForeignKey( country = models.ForeignKey(Country, models.DO_NOTHING, blank=True, null=True, to_field='id')
Country, models.DO_NOTHING, blank=True, null=True, to_field='id')
tags = models.TextField(blank=True, null=True) tags = models.TextField(blank=True, null=True)
description = models.TextField(blank=True, null=True) description = models.TextField(blank=True, null=True)
northing = models.TextField(blank=True, null=True) northing = models.TextField(blank=True, null=True)
easting = models.TextField(blank=True, null=True) easting = models.TextField(blank=True, null=True)
category = models.TextField( category = models.TextField(blank=True, null=True, choices=CATEGORY_CHOICES)
blank=True, null=True, choices=CATEGORY_CHOICES) subject = models.CharField(max_length=128, blank=True, null=True, choices=SUBJECT_CHOICES)
subject = models.CharField(
max_length=128, blank=True, null=True, choices=SUBJECT_CHOICES)
def __str__(self): def __str__(self):
if self.organisation_english: if self.organisation_english:
if self.organisation_2: if self.organisation_2:
name = f"{self.organisation_english} ({self.organisation_2})" name = f"{self.organisation_english} ({self.organisation_2})"
name = f"{self.country} < {self.organisation_english} < {self.organisation_2} < {self.organisation_3} < {self.acronym}"
else: else:
name = f"{self.organisation_english}" name = f"{self.organisation_english}"
name = f"{self.country} < {self.organisation_english} < {self.organisation_2} < {self.organisation_3} < {self.acronym}"
else: else:
name = '---' name = '---'
return name return name
class RangeOnlineInfo(models.Model): class RangeOnlineInfo(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
# This is supposed to be linked to MountainRange via range_name_map, but the range_name_map # This is supposed to be linked to MountainRange via range_name_map, but the range_name_map
@ -476,10 +450,8 @@ class RangeOnlineInfo(models.Model):
class RangeNameTranslation(models.Model): class RangeNameTranslation(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
range_name = models.ForeignKey( range_name = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True) language_translation = models.ForeignKey(Language, models.DO_NOTHING, blank=True, null=True)
language_translation = models.ForeignKey(
Language, models.DO_NOTHING, blank=True, null=True)
range_name_translation = models.TextField(blank=True, null=True) range_name_translation = models.TextField(blank=True, null=True)
def __str__(self): def __str__(self):
@ -532,10 +504,8 @@ class Keyword(models.Model):
class ResourceKeyword(models.Model): class ResourceKeyword(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
resource = models.ForeignKey( resource = models.ForeignKey(Resource, models.DO_NOTHING, blank=True, null=True)
Resource, models.DO_NOTHING, blank=True, null=True) keyword = models.ForeignKey(Keyword, models.DO_NOTHING, blank=True, null=True, to_field='keyword_id')
keyword = models.ForeignKey(
Keyword, models.DO_NOTHING, blank=True, null=True, to_field='keyword_id')
def __str__(self): def __str__(self):
return '%s - %s' % (self.resource.title, self.keyword.keyword) return '%s - %s' % (self.resource.title, self.keyword.keyword)
@ -558,10 +528,8 @@ class NamesImport(models.Model):
class ResourceRange(models.Model): class ResourceRange(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
resource_title = models.ForeignKey( resource_title = models.ForeignKey(Resource, models.DO_NOTHING, blank=True, null=True)
Resource, models.DO_NOTHING, blank=True, null=True) range_name = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
range_name = models.ForeignKey(
MountainRange, models.DO_NOTHING, blank=True, null=True)
def __str__(self): def __str__(self):
return '%s, %s' % (self.resource_title.title, self.range_name.name) return '%s, %s' % (self.resource_title.title, self.range_name.name)
@ -569,8 +537,7 @@ class ResourceRange(models.Model):
class LanguageLink(models.Model): class LanguageLink(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
language_number_code = models.ForeignKey( language_number_code = models.ForeignKey(Language, models.DO_NOTHING, blank=True, null=True)
Language, models.DO_NOTHING, blank=True, null=True)
language_letter_code = models.TextField(blank=True, null=True) language_letter_code = models.TextField(blank=True, null=True)
def __str__(self): def __str__(self):
@ -592,10 +559,8 @@ class Peak(models.Model):
class Search(models.Model): class Search(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
range_name = models.ForeignKey( range_name = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True) repository = models.ForeignKey(Repository, models.DO_NOTHING, blank=True, null=True)
repository = models.ForeignKey(
Repository, models.DO_NOTHING, blank=True, null=True)
search_string = models.TextField(blank=True, null=True) search_string = models.TextField(blank=True, null=True)
search_date = models.TextField(blank=True, null=True) search_date = models.TextField(blank=True, null=True)
result = models.TextField(blank=True, null=True) result = models.TextField(blank=True, null=True)
@ -610,7 +575,9 @@ class Person(models.Model):
MR_MRS_CHOICES = ( MR_MRS_CHOICES = (
('', ''), ('', ''),
('Ms', 'Ms'), ('Ms', 'Ms'),
('Ms.', 'Ms.'),
('Mr', 'Mr'), ('Mr', 'Mr'),
('Mr.', 'Mr.'),
) )
GMBA_FUNCTION_CHOICES = ( GMBA_FUNCTION_CHOICES = (
('', ''), ('', ''),
@ -634,9 +601,7 @@ Involved scientist''')
contact_email = models.EmailField(blank=True, null=True) # email_1 contact_email = models.EmailField(blank=True, null=True) # email_1
personal_url = models.URLField(blank=True, null=True) # url ? personal_url = models.URLField(blank=True, null=True) # url ?
biography = models.TextField(blank=True, null=True) biography = models.TextField(blank=True, null=True)
# Field renamed because it started with '_'. field_indexer = models.TextField(db_column='_indexer', blank=True, null=True) # Field renamed because it started with '_'.
field_indexer = models.TextField(
db_column='_indexer', blank=True, null=True)
mr_mrs = models.TextField(blank=True, null=True, choices=MR_MRS_CHOICES) mr_mrs = models.TextField(blank=True, null=True, choices=MR_MRS_CHOICES)
full_name = models.CharField(blank=True, null=True, max_length=100) full_name = models.CharField(blank=True, null=True, max_length=100)
@ -647,16 +612,12 @@ Involved scientist''')
mobile_number = models.CharField(blank=True, null=True, max_length=50) mobile_number = models.CharField(blank=True, null=True, max_length=50)
url = models.URLField(blank=True, null=True) url = models.URLField(blank=True, null=True)
field_of_expertise = models.TextField(blank=True, null=True) field_of_expertise = models.TextField(blank=True, null=True)
status = models.ForeignKey( status = models.ForeignKey(PeopleStatus, models.DO_NOTHING, blank=True, null=True, to_field='id')
PeopleStatus, models.DO_NOTHING, blank=True, null=True, to_field='id')
entry_date = models.TextField(blank=True, null=True) entry_date = models.TextField(blank=True, null=True)
gmba_function = models.TextField( gmba_function = models.TextField(blank=True, null=True, choices=GMBA_FUNCTION_CHOICES)
blank=True, null=True, choices=GMBA_FUNCTION_CHOICES)
news_letter = models.BooleanField(default=False) news_letter = models.BooleanField(default=False)
country = models.ForeignKey( country = models.ForeignKey(Country, models.DO_NOTHING, blank=True, null=True, to_field='id')
Country, models.DO_NOTHING, blank=True, null=True, to_field='id') organization = models.ForeignKey(Organisation, models.DO_NOTHING, blank=True, null=True)
organization = models.ForeignKey(
Organisation, models.DO_NOTHING, blank=True, null=True)
birds = models.BooleanField(default=False) birds = models.BooleanField(default=False)
mammals = models.BooleanField(default=False) mammals = models.BooleanField(default=False)
reptiles = models.BooleanField(default=False) reptiles = models.BooleanField(default=False)
@ -681,8 +642,7 @@ Involved scientist''')
assessment = models.BooleanField(default=False) assessment = models.BooleanField(default=False)
meta_analysis = models.BooleanField(default=False) meta_analysis = models.BooleanField(default=False)
synthesis = models.BooleanField(default=False) synthesis = models.BooleanField(default=False)
# "Qualitative social science methods (interviews, surveys)" qualitative_ssm = models.BooleanField(default=False) #"Qualitative social science methods (interviews, surveys)"
qualitative_ssm = models.BooleanField(default=False)
genetic_analyses = models.BooleanField(default=False) genetic_analyses = models.BooleanField(default=False)
field_site = models.BooleanField(default=False) field_site = models.BooleanField(default=False)
transect = models.BooleanField(default=False) transect = models.BooleanField(default=False)
@ -707,18 +667,10 @@ Involved scientist''')
return False return False
def index(self): def index(self):
if self:
self.field_indexer = " ".join([ self.field_indexer = " ".join([
self.first_name if self and self.first_name else "", self.first_name, self.last_name, self.organisation, self.position, self.biography
self.last_name if self and self.last_name else "", ])
self.organisation if self and self.organisation else "",
self.position if self and self.position else "",
self.biography if self and self.biography else ""
] if self else "")
# self.save()
return True return True
else:
return False
def fullname(self): def fullname(self):
return " ".join([self.title if self.title else '', return " ".join([self.title if self.title else '',
@ -740,16 +692,11 @@ Involved scientist''')
'biography': self.biography or '', 'biography': self.biography or '',
} }
def save(self, *args, **kwargs):
self.index()
super(Person, self).save(*args, **kwargs)
class TaxaPeople(models.Model): class TaxaPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
taxon = models.ForeignKey( taxon = models.ForeignKey('Taxon', models.DO_NOTHING, blank=True, null=True)
'Taxon', models.DO_NOTHING, blank=True, null=True)
class Meta: class Meta:
db_table = 'taxa_people' db_table = 'taxa_people'
@ -761,8 +708,7 @@ class TaxaPeople(models.Model):
class RangesPeople(models.Model): class RangesPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
range = models.ForeignKey( range = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True)
class Meta: class Meta:
db_table = 'ranges_people' db_table = 'ranges_people'
@ -774,8 +720,7 @@ class RangesPeople(models.Model):
class ResourcesPeople(models.Model): class ResourcesPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
resource = models.ForeignKey( resource = models.ForeignKey(Resource, models.CASCADE, blank=True, null=True)
Resource, models.CASCADE, blank=True, null=True)
class Meta: class Meta:
db_table = 'resources_people' db_table = 'resources_people'
@ -786,8 +731,7 @@ class ResourcesPeople(models.Model):
class ScalesPeople(models.Model): class ScalesPeople(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey( person = models.ForeignKey(Person, models.DO_NOTHING, blank=True, null=True)
Person, models.DO_NOTHING, blank=True, null=True)
scale = models.ForeignKey(Scale, models.DO_NOTHING, blank=True, null=True) scale = models.ForeignKey(Scale, models.DO_NOTHING, blank=True, null=True)
class Meta: class Meta:
@ -811,8 +755,7 @@ class Species(models.Model):
class PeopleFunction(models.Model): class PeopleFunction(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
function = models.ForeignKey( function = models.ForeignKey(GMBA_function, models.DO_NOTHING, blank=True, null=True)
GMBA_function, models.DO_NOTHING, blank=True, null=True)
def __str__(self): def __str__(self):
return self.person.full_name return self.person.full_name
@ -820,10 +763,8 @@ class PeopleFunction(models.Model):
class SpeciesRange(models.Model): class SpeciesRange(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
scientific_name = models.ForeignKey( scientific_name = models.ForeignKey(Species, models.DO_NOTHING, blank=True, null=True)
Species, models.DO_NOTHING, blank=True, null=True) range = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
range = models.ForeignKey(
MountainRange, models.DO_NOTHING, blank=True, null=True)
endemic = models.TextField(blank=True, null=True) endemic = models.TextField(blank=True, null=True)
source_url = models.TextField(blank=True, null=True) source_url = models.TextField(blank=True, null=True)
@ -834,8 +775,7 @@ class SpeciesRange(models.Model):
class PeopleRange(models.Model): class PeopleRange(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
range = models.ForeignKey( range = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True)
def __str__(self): def __str__(self):
return str(self.id) return str(self.id)
@ -843,8 +783,7 @@ class PeopleRange(models.Model):
class TaxonRange(models.Model): class TaxonRange(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
range = models.ForeignKey( range = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True)
taxon = models.TextField(blank=True, null=True) taxon = models.TextField(blank=True, null=True)
subrange_or_region = models.TextField(blank=True, null=True) subrange_or_region = models.TextField(blank=True, null=True)
taxon_status = models.TextField(blank=True, null=True) taxon_status = models.TextField(blank=True, null=True)
@ -875,8 +814,7 @@ class PeopleResource(models.Model):
) )
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True) person = models.ForeignKey(Person, models.CASCADE, blank=True, null=True)
resource = models.ForeignKey( resource = models.ForeignKey(Resource, models.DO_NOTHING, blank=True, null=True)
Resource, models.DO_NOTHING, blank=True, null=True)
role = models.TextField(blank=True, null=True, choices=ROLES_CHOICES) role = models.TextField(blank=True, null=True, choices=ROLES_CHOICES)
def __str__(self): def __str__(self):
@ -885,10 +823,8 @@ class PeopleResource(models.Model):
class RangeCountry(models.Model): class RangeCountry(models.Model):
id = models.AutoField(primary_key=True) id = models.AutoField(primary_key=True)
range = models.ForeignKey( range = models.ForeignKey(MountainRange, models.DO_NOTHING, blank=True, null=True)
MountainRange, models.DO_NOTHING, blank=True, null=True) country = models.ForeignKey(Country, models.DO_NOTHING, blank=True, null=True)
country = models.ForeignKey(
Country, models.DO_NOTHING, blank=True, null=True)
def __str__(self): def __str__(self):
return '%s --- %s' % (self.range.range_name if self.range else 'None', self.country.short_name if self.country else 'None') return '%s --- %s' % (self.range.range_name if self.range else 'None', self.country.short_name if self.country else 'None')

11
app/static/app/gmba-search.tag
View file

@ -248,7 +248,7 @@
</div><!-- /person-tags --> </div><!-- /person-tags -->
<a name="contact"></a><h3>Contact</h3> <a name="contact"></a><h3>Contact</h3>
<form onsubmit="{ FormSubmitted }" action="https://formspree.io/f/xvolqjpe" <form action="https://formspree.io/&#103;&#109;&#098;&#097;&#064;&#105;&#112;&#115;&#046;&#117;&#110;&#105;&#098;&#101;&#046;&#099;&#104;"
target="_blank" method="POST" class="contact-form"> target="_blank" method="POST" class="contact-form">
<input type="hidden" name="subject" value="Contact request from GMBA Connect"> <input type="hidden" name="subject" value="Contact request from GMBA Connect">
<input type="hidden" name="person" value={ person.data.fullname }> <input type="hidden" name="person" value={ person.data.fullname }>
@ -287,12 +287,6 @@
const FILTER_BLANK = { const FILTER_BLANK = {
'country': [], 'range': [], 'field': [], 'taxon': [] 'country': [], 'range': [], 'field': [], 'taxon': []
} }
FormSubmitted(){
console.log("i got executed")
var self = this
self.closedetails()
}
this.filters_shown = FILTER_BLANK this.filters_shown = FILTER_BLANK
search(e, nextpage) { search(e, nextpage) {
@ -487,9 +481,6 @@
} }
}) })
</script> </script>
</gmba-search> </gmba-search>

2
app/templates/app/admin/base.html
View file

@ -9,8 +9,6 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content=""> <meta name="description" content="">
<meta name="author" content=""> <meta name="author" content="">
<meta name="referrer" content="origin">
<link href="{% static 'app/admin/bootstrap/bootstrap3/swatch/default/bootstrap.min.css' %}?v=3.3.5" rel="stylesheet"> <link href="{% static 'app/admin/bootstrap/bootstrap3/swatch/default/bootstrap.min.css' %}?v=3.3.5" rel="stylesheet">

2
app/templates/app/index.html
View file

@ -6,8 +6,6 @@
<meta charset="utf-8" /> <meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="referrer" content="origin">
<!--[if lte IE 8]><script src="/static/assets/js/ie/html5shiv.js"></script><![endif]--> <!--[if lte IE 8]><script src="/static/assets/js/ie/html5shiv.js"></script><![endif]-->
<!--[if lte IE 8]><link rel="stylesheet" href="/static/assets/css/ie8.css" /><![endif]--> <!--[if lte IE 8]><link rel="stylesheet" href="/static/assets/css/ie8.css" /><![endif]-->

2
app/templates/app/offline.html
View file

@ -5,8 +5,6 @@
<meta charset="utf-8" /> <meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="referrer" content="origin">
<!--[if lte IE 8]><script src="{% static 'app/assets/js/ie/html5shiv.js' %}"></script><![endif]--> <!--[if lte IE 8]><script src="{% static 'app/assets/js/ie/html5shiv.js' %}"></script><![endif]-->
<!--[if lte IE 8]><link rel="stylesheet" href="{% static 'app/assets/ss/ie8.css' %}" /><![endif]--> <!--[if lte IE 8]><link rel="stylesheet" href="{% static 'app/assets/ss/ie8.css' %}" /><![endif]-->

16
app/views.py
View file

@ -176,17 +176,11 @@ def get_paginated(query_set, page, per_page):
for p in paginator.object_list: for p in paginator.object_list:
filters['country'].append(p.country.short_name) filters['country'].append(p.country.short_name)
for r in p.peoplerange_set.all(): for r in p.peoplerange_set.all():
if r.range and r.range.range_name:
filters['range'].append(r.range.range_name) filters['range'].append(r.range.range_name)
else:
print("r.range.range_name is None %s %s" % (r.range_id, r.person_id))
for r in p.fieldspeople_set.all(): for r in p.fieldspeople_set.all():
filters['field'].append(r.field.name) filters['field'].append(r.field.name)
for r in p.taxapeople_set.all(): for r in p.taxapeople_set.all():
if r and r.taxon and r.taxon.name:
filters['taxon'].append(r.taxon.name) filters['taxon'].append(r.taxon.name)
else:
print("r.taxon.name is None %s %s" % (r.range_id, r.taxon))
filters = { filters = {
'country': sorted(set(filters['country'])), 'country': sorted(set(filters['country'])),
'range': sorted(set(filters['range'])), 'range': sorted(set(filters['range'])),
@ -213,10 +207,10 @@ class SearchView(View):
per_page = int(self.request.GET.get('per_page', 10)) per_page = int(self.request.GET.get('per_page', 10))
q = self.request.GET.get('q', '').strip() q = self.request.GET.get('q', '').strip()
if not q or len(q) < 3: if not q or len(q) < 3:
query_set = Person.objects.filter(profile_on_web=True).all() query_set = Person.objects.all()
else: else:
query = reduce(operator.or_, (Q(field_indexer__icontains=item) for item in q.split(" "))) query = reduce(operator.or_, (Q(field_indexer__icontains=item) for item in q.split(" ")))
query_set = Person.objects.filter(query).filter(profile_on_web=True) query_set = Person.objects.filter(query)
q_country = self.request.GET.get('country', '') q_country = self.request.GET.get('country', '')
q_range = self.request.GET.get('range', '') q_range = self.request.GET.get('range', '')
@ -225,7 +219,7 @@ class SearchView(View):
if len(q_country) > 2: if len(q_country) > 2:
query_set = query_set.filter(country__short_name__icontains=q_country.strip().lower()) query_set = query_set.filter(country__short_name__icontains=q_country.strip().lower())
if len(q_range) > 2: if len(q_range) > 2:
ranges_people = PeopleRange.objects.filter(range__range_name__icontains=q_range.strip().lower()) ranges_people = PeopleRange.objects.filter(range__name__icontains=q_range.strip().lower())
r_people_ids = [rp.person_id for rp in ranges_people] r_people_ids = [rp.person_id for rp in ranges_people]
query_set = query_set.filter(id__in=r_people_ids) query_set = query_set.filter(id__in=r_people_ids)
if len(q_field) > 2: if len(q_field) > 2:
@ -245,8 +239,8 @@ class PeopleDetailView(View):
person = get_object_or_404(Person, id=people_id) person = get_object_or_404(Person, id=people_id)
return_data = { return_data = {
'data': person.dict(), 'data': person.dict(),
'resources': [r.resource.dict() for r in person.peopleresource_set.all()], 'resources': [r.resource.dict() for r in person.resourcespeople_set.all()],
'ranges': [r.range.dict() for r in person.peoplerange_set.all()], 'ranges': [r.range.dict() for r in person.rangespeople_set.all()],
'fields': [r.field.name for r in person.fieldspeople_set.all()], 'fields': [r.field.name for r in person.fieldspeople_set.all()],
'methods': [r.method.name for r in person.methodspeople_set.all()], 'methods': [r.method.name for r in person.methodspeople_set.all()],
'scales': [r.scale.name for r in person.scalespeople_set.all()], 'scales': [r.scale.name for r in person.scalespeople_set.all()],

37
docker-compose.prod.yml
View file

@ -1,37 +0,0 @@
version: '3.8'
services:
web:
build:
context: ./
dockerfile: Dockerfile.prod
command: gunicorn gmba_django.wsgi:application --bind [::]:8000
volumes:
- static_volume:/data/app/app/static
- media_volume:/data/app/app/media
expose:
- 8000
env_file:
- ./.env.prod
depends_on:
- db
db:
image: postgres:13.0-alpine
volumes:
- postgres_data:/data/var/lib/postgresql/data/
env_file:
- ./.env.prod
nginx:
build: ./nginx
volumes:
- static_volume:/data/app/app/static
- media_volume:/data/app/app/media
ports:
- 1337:80
depends_on:
- web
volumes:
postgres_data:
static_volume:
media_volume:

15
entrypoint.prod.sh
View file

@ -1,15 +0,0 @@
#!/bin/sh
if [ "$DATABASE" = "postgres" ]
then
echo "Waiting for postgres..."
while ! nc -z $SQL_HOST $SQL_PORT; do
sleep 0.1
done
echo "PostgreSQL started"
fi
exec "$@"

8
gmba_django/settings/development.py
View file

@ -4,11 +4,7 @@ DEBUG = True
DATABASES = { DATABASES = {
'default': { 'default': {
'ENGINE': os.environ.get('SQL_ENGINE', 'django.db.backends.postgresql'), 'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.environ.get('POSTGRES_DB', 'app'), 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'USER': os.environ.get('POSTGRES_USER', 'app'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD', ''),
'HOST': os.environ.get('POSTGRES_HOST', 'localhost'),
'PORT': os.environ.get('POSTGRES_PORT', 5432),
} }
} }

15
gmba_django/settings/production.py
View file

@ -6,20 +6,17 @@ ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', "").split(",")
DATABASES = { DATABASES = {
'default': { 'default': {
'ENGINE': os.environ.get('SQL_ENGINE', 'django.db.backends.postgresql'), 'ENGINE': os.environ.get('SQL_ENGINE', 'django.db.backends.sqlite3'),
'NAME': os.environ.get('POSTGRES_DB', 'app'), 'NAME': os.environ.get('SQL_DATABASE', os.path.join(BASE_DIR, 'db.sqlite3')),
'USER': os.environ.get('POSTGRES_USER', 'app'), # 'USER': os.environ.get('SQL_USER', 'user'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD', ''), # 'PASSWORD': os.environ.get('SQL_PASSWORD', 'password'),
'HOST': os.environ.get('POSTGRES_HOST', 'localhost'), # 'HOST': os.environ.get('SQL_HOST', 'localhost'),
'PORT': os.environ.get('POSTGRES_PORT', 5432), # 'PORT': os.environ.get('SQL_PORT', ''),
} }
} }
SECURE_SSL_REDIRECT = True SECURE_SSL_REDIRECT = True
# See https://docs.djangoproject.com/en/2.2/ref/settings/#secure-proxy-ssl-header
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True CSRF_COOKIE_SECURE = True

6
init.sh
View file

@ -1,6 +0,0 @@
#!/bin/ash
#python manage.py migrate
python manage.py collectstatic --noinput
#
#gunicorn gmba_django.wsgi:application --bind [::]:8000

4
nginx/Dockerfile
View file

@ -1,4 +0,0 @@
FROM nginx:1.21-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d

28
nginx/nginx.conf
View file

@ -1,28 +0,0 @@
upstream gmba_django {
server web:8000;
}
server {
listen 80;
client_max_body_size 256m;
location / {
proxy_pass http://gmba_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /home/app/app/static/;
expires max;
}
location /media/ {
alias /home/app/app/media/;
expires max;
}
add_header Content-Security-Policy "frame-ancestors https://www.gmba.unibe.ch";
}

3
requirements.txt
View file

@ -1,9 +1,8 @@
asgiref==3.4.1 asgiref==3.4.1
Django==4.0.1 Django==3.2.5
python-dotenv==0.19.0 python-dotenv==0.19.0
pytz==2021.1 pytz==2021.1
sqlparse==0.4.1 sqlparse==0.4.1
typing-extensions==3.10.0.0 typing-extensions==3.10.0.0
psycopg2==2.9.1 psycopg2==2.9.1
Werkzeug==2.0.2 Werkzeug==2.0.2
gunicorn==20.1.0