How to give input to a website using python

hi all I am new to python. please help me with this requirement.

http://www.example.com/ratings/ratings-rationales.jsp?date=true&result=true

In this link, I have to choose date first, then the rating company will list its publications as links. Now i wanted to search a link that contains a word of my interest say "stable". I have tried the following using python 3.4.2

from bs4 import BeautifulSoup
from urllib.parse import urljoin
import requests

url = "http://www.example.com/ratings/ratings-rationales.jsp?date=true&result=true"   
r = requests.get(url)
soup = BeautifulSoup(r.content)

example_links = lambda tag: getattr(tag, 'name', None) == 'a' and 'stable' in tag.get_text().lower() and 'href' in tag.attrs
results = soup.find_all(example_links)
result_links = [urljoin(url, tag['href']) for tag in results]
print (result_links)

This is not printing anything. Iam seeing below as result

>>>
[]

Obviously Iam not giving date as input.
1. How to input from and to dates as today's date ? (Obviously to check periodically for updates of the links containing a word of interest, which will be question for later time)
For example after giving from date: 31-12-2014 to date: 31-12-2014 as inputs

is the output I need as hyperlink.

Any suggestion will be much useful. Thanks in advance

Here is the updated code still Iam not able to get the result. >>> [] is the output

from datetime import datetime
from bs4 import BeautifulSoup
from urllib.parse import urljoin
import requests

#Getting the current date
today = datetime.today()

#For the sake of brevity some parameters are missing on the payload
payload = {
    'selArchive': 1,
    'selDay': 31, 
    'selMonth': 12, 
    'selYear': 2014,
    'selDay1': 31, 
    'selMonth2': 12, 
    'selYear1': 2014,
    'selSector': '',
    'selIndustry': '',
    'selCompany': ''
}

example_url = "http://www.example.com/
r = requests.post(example_url, data=payload)    
rg = requests.get(example_url)
soup = BeautifulSoup(rg.content)

crisil_links = lambda tag: getattr(tag, 'name', None) == 'a' and 'stable' in tag.get_text().lower() and 'href' in tag.attrs   
results = soup.find_all(example_links)
result_links = [urljoin(url, tag['href']) for tag in results]
print (result_links)

Here’s a walkthrough of a website I built that will analyze text sentiment on the fly

All data scientists know how good computers are getting at analysing text, but most other people don’t. So being a man of the people, I thought I’d give them what they don’t know they need — a website where they can analyse their own eBook.

You can read about the usefulness of the website here. But this article is aimed more at all you techy lot and centres around how to run python scripts on user input and produce an output — without saving the input anywhere.

This is super useful as it's completely automated, as long as the servers don’t crash (they might — I have no idea) then this should just run itself. No maintenance whatsoever.

The background

So the aim of the game was to produce a website that takes user input, runs python scrips on them, and then returns them to the user in a usable fashion. Turns out it's actually not that easy to run Python scripts online and I didn’t find much help online — but below is all that I found.

The best approach I found was to use Flask — a micro web framework written in Python. Flask, along with Django, is commonly used in web development. And since they are written in Python, it is easy to integrate Python scripts into your routing.

I’ve written a pretty popular piece on how to show Matplotlib plots and Panda’s dataframes before — if that sounds useful then please check it out.

So as a reminder the goal here is to take a user input from an HTML form, run a python script on it and output the result in a fully automatic process.

Set up

The setup for this project is pretty straightforward. We first have to import all the required packages from Flask, which can be seen below (Line 2). We then initialise the app (line 5) — you can call it whatever you want but it's a convention to use dunder ‘name’. Just don’t call it Flask as this will conflict with Flask itself. And finally, we use the route decorator to tell Flask which URL should call our function. In this case, it's simply ‘/’ — so our initial page.

As can be seen, we have returned the ‘render_template’ function which is essential to any Flask app. It returns an HTML file from the templates folder (Flask requires a specific templates folder) and renders it on the specified page. So here’s the HTML we will need.

As can be seen, it's a pretty straightforward HTML page as you’ll have seen one hundred times before. Lines 11–15 are the ones of interest to us, however. In line 11 we specify the beginning of our form and close it at line 15. Within the form, we have three div tags (lines 12,13, & 14). Line 12 is simply the form title, which is nothing more than text, line 13 is the file selection and line 14 is the submit button. As you can see here, we’ve specified the input type as “file” and specifically stated .txt files.

It’s important here to state “enctype= ‘multipart/form-data”, so Flask knows how to encode the input. Even more importantly don’t forget to state method = “POST” so Flask knows to request the information stored in the form.

Running the Python scripts on the inputs.

Now that we have the barebones of our site up and running, how do we run the python scripts on the inputs and then show the user the results?

What we need to do is, create another decorator that runs only when receiving post requests.

Which is exactly what the below does. You can pop this in directly under the original route decorator.

So, here, when the user clicks the submit button this function is triggered as it is activated when POST methods are used. Line 5 then uses the request library to select the inputted file. You can then simply run your python scripts on this variable as you normally would in Python. Once you have done the manipulation you can then return the variables to the user in one of many fashions. You can use the Response method as I did above, or you could pass it to a variable and output it directly in another HTML template using render_template() again. The possibilities are endless.

You can host your website in numerous ways, but I prefer PythonAnywhere. A guide on how to do it can be seen at the end of my article here.

As mentioned, you can see a working example at www.ebookanalyzer.com.

I’m certainly no web developer so excuse the ugliness.

Also please let me know if there are any bugs or you can see any improvements or more elegant solutions.

If I’ve inspired you to join medium I would be really grateful if you did it through this link — it will help to support me to write better content in the future.If you want to learn more about data science, become a certified data scientist, or land a job in data science, then checkout 365 data science through my affiliate link.

If you enjoyed this, here are some other similar articles I wrote:

Cheers,

James

How do you enter data into a website using Python?

How do I put input on my website?

Can you use Python to code a website?