Upgrade to Python 3.11 on Ubuntu 22.04 LTS

My Ubuntu 22.04 (WSL) comes with Python 3.10.6, and I need to upgrade it to 3.11 for a workshop. (More importantly is, it claims to be 10-60% faster than the previous 3.10. 😎

Here are the steps:

$ sudo add-apt-repository ppa:deadsnakes/ppa

$ sudo apt update  

$ sudo apt install python3.11-full

$ python3.11 --version
Python 3.11.1

Next. To set Python 3.11 as default.

$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 110

$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 100

$ sudo update-alternatives --config python3

Upgrade Ubuntu to 22.04 LTS Jammy Jellyfish

I have Ubuntu 20.04 installed under Windows Subsystem for Linux (WSL).

Below are the steps to upgrade Ubuntu to 22.04 Jammy Jellyfish (so that that I can enjoy my python3.10).

$ sudo apt update
$ sudo apt upgrade
$ sudo apt dist-upgrade

$ sudo apt autoremove

$ sudo apt install update-manager-core

$ sudo do-release-upgrade 

Follow by a couple of 'y' 

The whole process took around 2 hours to finish.

Links:

• How To Upgrade Ubuntu To 22.04 LTS Jammy Jellyfish
  

Upgrade to Python 3.10 on Ubuntu 20.04 LTS

My Ubuntu 20.04 comes with Python 3.8.10, and I need to upgrade it to 3.10 for a workshop.

Here are the steps:

$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt-get update

$ apt-get update

$ apt list | grep python3.10

$ sudo apt-get install python3.10

$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 2

$ sudo update-alternatives --config python3

$ python3 -V

Patch_Tuesday Utils

A step-by-step how I generate patch_tuesday.exe from python script. You can download the python script or the executable directly (at GitHub).

c:\home> pip install colorama

c:\home> pip install pyfiglet

c:\home> pip install pyinstaller

 

Method 1:

c:\home> pyinstaller --onefile --add-data "C:\Python3\Lib\site-packages\pyfiglet";./pyfiglet   patch_tuesday.py

Method 2:

c:\home> pyinstaller --onefile --collect-all pyfiglet patch_tuesday.py

c:\home> cd dist

c:\home\dist> patch_tuesday.exe -h

patch_tuesday.exe -h

c:\home\dist> patch_tuesday.exe -k 2022-jun -v

patch_tuesday.exe -k 2022-jun -v

Tips: Try "patch_tuesday.exe -k 2022-jun -vc" for chart output.

Links:

• MS Patch Tuesday (MSRFC - Security Update Guide)

Invert a Complex Dictionary in Python

Have you ever need to invert a dictionary with value being a list in Python?

Here is what I mean. We need to convert the following dict:

{ "apple"      : [ "green", "red" ], 

  "watermelon" : [ "green" ], 

  "strawberry" : [ "red" ], 

  "lemon"      : [ "green", "yellow" ] }

To a new dict as below:

{ 'green'  : ['apple', 'watermelon', 'lemon'], 

  'red'    : ['apple', 'strawberry'], 

  'yellow' : ['lemon'] } 

Here's my solution which using the defaultdict from collections:

>>> from collections import defaultdict

>>> a_dict = { "apple" : [ "green", "red" ], "watermelon" : [ "green" ], "strawberry" : [ "red" ], "lemon" : [ "green", "yellow" ] }

>>> b_dict = defaultdict(list)

>>> for k,v in a_dict.items():

...    for k1 in v:

...        b_dict[k1].append(k)

...

>>> b_dict

defaultdict(<class 'list'>, {'green': ['apple', 'watermelon', 'lemon'], 'red': ['apple', 'strawberry'], 'yellow': ['lemon']})

There is another way that simplies what we have above:

>>> from collections import defaultdict

>>> a_dict = { "apple" : [ "green", "red" ], "watermelon" : [ "green" ], "strawberry" : [ "red" ], "lemon" : [ "green", "yellow" ] }

>>> b_dict = defaultdict(list)

>>> { b_dict[k1].append(k) for k,v in a_dict.items() for k1 in v }

>>> b_dict

Alright, hope this helps!!

Python and GitHub Secrets

This is a simple note to show how to access GitHub Secrets with Python.

First, to add a new secret, go to GitHub repository > Settings > Secrets > New Repository Secret.

Second, define a Name ('SEC_NAME') and put in the Value.

 

Next, map them as environment variables in GitHub Actions Workflow.

....

    - name: Run tests

        env:

            API_KEY: ${{ secrets.SEC_NAME }}

        run:  |

....

 

Finally, refer to the env variable in Python script.

import os

API_KEY = os.environ['API_KEY']

....

Print Colors And Formatted Text in Terminal

This is my note on how to print formatted text and colors in terminal using Python. There are several methods to output colored text to terminal.

Terminal can be so plain. And formatted text (aka styled text or rich text) can be used as opposed to plan text, has styling info like:

• color (text and background)
• style (bold and italic)
• others (strike-through, underline)

 

1. Using ANSI Escape Codes

ANSI escape codes are used to control the formatting and color in Linux terminal. To encode this formatting info, certain sequences of bytes are embedded into the text, which the terminal looks for and interprets as commands and executes them.

print('\x1b[3;31;43m' + 'Hello world!' + '\x1b[0m')

The python statement above will produce the output in "red" text-color, with "yellow" background, and "italic" style.

The general syntax is \x1b[A;B;C   and the first ANSI escape code used is  \x1b[3;31;43m  . The second ANSI escape code used is \x1b[0m , and this is the code used to reset the color/style to defaults values. 

A is for text formatting style, ranges from 1 to 9.

• 1 : bold
• 2 : faint
• 3 : italic
• 4 : underline
• 5 : blinking
• 6 : fast blinking
• 7 : reverse
• 8 : hide
• 9 : strikethrough
  

B is for text color, ranges from 30 t0 37.

C is for background color, ranges from 40 to 47.

• 30/40 : black
• 31/41 : red
• 32/42 : green
• 33/43 : yellow
• 34/44 : blue
• 35/45 : magenta
• 36/46 : cyan
• 37/47 : white

2. Using built-in modules : colorama, termcolor

Example of using colorama

from colorama import init, Fore, Back, Style

init (autoreset=True, strip=False)

print(Fore.RED + + Back.GREEN + Style.DIM + f'Red text on Green in dim')

print(f'Normal')

 

Example of using termcolor

>>> from termcolor import colored
>>> print(colored('Hello, World!', 'green', 'on_red'))

A more complex example:

# Python program to print
# colored text and background
import sys
from termcolor import colored, cprint

text = colored('Hello, World!', 'red', attrs=['reverse', 'blink'])
print(text)
cprint('Hello, World!', 'green', 'on_red')

print_red_on_cyan = lambda x: cprint(x, 'red', 'on_cyan')
print_red_on_cyan('Hello, World!')
print_red_on_cyan('Hello, Universe!')

for i in range(10):
    cprint(i, 'magenta', end=' ')

cprint("Attention!", 'red', attrs=['bold'], file=sys.stderr)

Links

• https://www.geeksforgeeks.org/print-colors-python-terminal/
• https://www.geeksforgeeks.org/formatted-text-linux-terminal-using-python/

KEV Dashboard

Just finished KEV Dashboard today.

Here is another simple python script on creating simple dashboard for CISA's Known Exploited Vulnerabilitiy (KEV).

There are 2 modes, text-based (default) and simple chart-based.

Text-based dashboard

Simple chart-based dashboard

The code will be uploaded to Github soon.

Publish kev-catalog on GitHub

The cisa-alerts.py script has been renamed to kev-catalog.py and been published to GitHub today. Just download, setup and run the script.

$ git clone https://github.com/myseq/kev-catalog

$ cd kev-catalog/

$ pip3 install -r requirements.txt

$ python3 kev-catalog.py -v 

kev-catalog.py

Links:

• https://github.com/myseq/kev-catalog
  

Update on cisa-alerts.py

cisa-alerts.py

Update on 'cisa-alerts.py'. 

• Search CVE within catalog.
• Search string within catalog.
• Specify the top N vendors and products.
• Specify the last N days of CVE added to catalog.

 

cisa-alerts.py -e 2017-0143

cisa-alerts.py -s keep

cisa-alerts.py -l 4 -i 6

Links:

• https://github.com/myseq/

Display Images on Terminal

This is easy way to display images on a terminal using Python. The module is called climage and has the following features:

• convert images to NASI Escape codes
• allow 8/16/256 bit color codings
• provide ASCII/Unicode support
  

Installation

$ pip install climage

Usage (in Python)

convert(filename, is_unicode=False, is_truecolor=False, is_256color=True, is_16color=False, is_8color=False, width=80, palette=”default”)

Parameters:
filename : Name of image file.
is_unicode :  If true, conversion is done in unicode format, otherwise ASCII characters will be used.
is_truecolor :  Whether to use RGB colors in generation, if supported by terminal. Defaults False.
is_256color : Whether to use 256 colors encoding. Defaults True.
is_16color : Whether to use 16 colors encoding. Defaults False.
is_8color : Whether to use first 8 System colors. Defaults False.
width : Number of blocks of console to be used. Defaults to 80.
palette : Sets mapping of RGB colors scheme to system colors. Options are : [“default”, “xterm”, “linuxconsole”, “solarized”, “rxvt”, “tango”, “gruvbox”, “gruvboxdark”]. Default is “default”.

 

to_file(infile, outfile, is_unicode=False, is_truecolor=False, is_256color=True, is_16color=False, is_8color=False, width=80, palette=”default”)

Parameters:
infile : The name/path of image file.
outfile :   File in which to store ANSI encoded string. 

Links:

• https://www.geeksforgeeks.org/display-images-on-terminal-using-python/
• https://pypi.org/project/climage/
  

Create Progress Bar in Terminal

This is easy way to create and show the progress bar in terminal using Python. This is useful for installation or loading a page. The module is called tqdm and it eases your mind with a small progress bar to show an estimation of a process.

Installation

$ pip install tqdm

Usage (in Python)

A simple example for testing

from tqdm import tqdm

 

for i in tqdm(range(int(9e6))):
    pass

Use a parameter to specify the description of progress bar.

from tqdm import tqdm
from time import sleep

for i in tqdm(range(0, 100), desc ="Loading"):
    sleep(.1)

To specify the total number of expected iterations.

from tqdm import tqdm
from time import sleep

for i in tqdm(range(0, 100), total = 500,desc ="Loading"):
    sleep(.1)

To specify the entire width.

from tqdm import tqdm
from time import sleep

for i in tqdm(range(0, 100), ncols = 100,desc ="Loading"): 

    sleep(.1)

To specify the minimum progress display update (default is 0.1 seconds).

from tqdm import tqdm
from time import sleep

for i in tqdm(range(0, 100), mininterval = 3, desc ="Loading"):
    sleep(.1)

To fill the progress bar with ASCII characters.

from tqdm import tqdm
from time import sleep

for i in tqdm(range(0, 100), ascii ="123456789$"):
    sleep(.1)

Links:

• https://www.geeksforgeeks.org/python-how-to-make-a-terminal-progress-bar-using-tqdm/
• https://pypi.org/project/tqdm/
  

7 Newly Added Known Exploited Vulnerabilities

cisa-alerts.py

 

With the update of the cisa-alerts script today, it can now show the details of what's new in CISA's Known Exploited vulnerabilites JSON file.

There are 7 newly added Known Exploited vulnerabilities, and 1 of them is for Microsoft Windows User Profile Service Privilege Escalation Vulnerability (cve-2021-34484). 

All the 7 vulnerabilities need to be patched by Apr 21, 2022 (20 days for remediation).

Links:

• https://www.cisa.gov/uscert/ncas/current-activity/2022/03/31/cisa-adds-seven-known-exploited-vulnerabilities-catalog
  

Update on Known Exploited Vulnerabilities

Quick update on known exploited vulnerabilities catalog by CISA.

• Total vulns : 602
• Overdue vulns : 247
• Upcoming vulns : 355
• Vendors : 120
• Products: 326
  

cisa-alerts.py

32 vulnerabilities are added to the list (19 from Microsoft). All the details is available for download at CISA. Please refer to the previous post.

(If you notice there is a extra .2 at the catalog version, it is from CISA)

2022.03.28.2

Links:

• https://myseq.blogspot.com/2022/03/cisa-known-exploited-vuln-catalog.html

CISA Known Exploited Vuln Catalog

On Nov 3, 2021, the Cybersecurity and Infrastructure Security Agency (CISA), a branch of the U.S. Department of Homeland Security (DHS), released Binding Operational Directive (BOD) 22-01. It is tend to be high-level and high-impact, and unusually direct to mitigate a specific list of vulnerabilities in a strict time frame.

CISA BOD 22-01 has three lines specific to patching requirements:

1. Remediate each vulnerability according to the timelines set forth in the CISA-managed vulnerability catalog.
2. The catalog will list exploited vulnerabilities that carry significant risk to the federal enterprise with the requirement to remediate within six months for vulnerabilities with a Common Vulnerabilities and Exposures (CVE) ID assigned prior to 2021 and within two weeks for all other vulnerabilities.
3. These default timelines may be adjusted in the case of grave risk to the Federal Enterprise.

Overall, this seems to follow patching guidance many commercial entities already use.This catalog is called Known Exploited Vulnerability (KEV) catalog and it is strongly recommends that everyone to review and monitor the catalog and remediate the listed vulnerabilities to strengthen their security and resilience posture.

CISA will update this catalog with additional exploited vulnerabilities as they become known, subject to CISA review and when they satisfy the following thresholds:

• The vulnerability has an assigned Common Vulnerabilities and Exposures (CVE) ID.
• There is reliable evidence that the vulnerability has been actively exploited in the wild.
• There is a clear remediation action for the vulnerability, such as a vendor provided update.

cisa-alerts.py

 

The simple python script shows the top-n vendors and the top-n products found in the json file. It also can show the Kenna query string on CVE, with overdue and upcoming CVE.

Links:

• https://cyber.dhs.gov/bod/22-01/
  
• https://www.cisa.gov/known-exploited-vulnerabilities-catalog
• https://github.com/myseq/
  

CodExt – Python Codecs extension

CodExt is a (Python2-3 compatible) library that extends the native codecs library (namely for adding new custom encodings and character mappings), hence its name combining CODecs EXTension. It provides 120+ new codecs and also features a guessing mode for decoding multiple layers of encoding and CLI tools for convenience.

Links:

• https://github.com/dhondta/python-codext
• https://hakin9.org/codext-encode-decode-anything-with-python/
  

Introducing PyFiglet

PyFiglet is a full port of FIGlet (http://www.figlet.org/) into pure python. It takes ASCII text and renders it in ASCII art fonts (like the title above, which is the 'block' font).

Below is the simple one-liner to export all the fonts pyfiglet supporting into a file (for reference later).

$ pyfiglet -l

$ for FONTS in `pyfiglet -l`; do echo $FONTS >> fonts ; pyfiglet $FONTS -f $FONTS >> fonts ; done;

Mouse Jiggler

With the idea from HakByte, we wrote a similar small python utils and convert it o Windows EXE. 

This is mouse jiggler tool which prevents it from falling asleep by moving the mouse to simulate human input (bypass screensaver locking).

Then I just convert it to Windows executable with pyinstaller.

pyinstaller --onefile --noconsole mouse_jiggler.py

pyinstaller --onefile -w -f  mouse_jiggler.py

Links:

• Mouse Jiggler - https://github.com/myseq/utils
  

Run Python Code on WhatsApp

Ever thought of running simple python code on your WhatsApp (mobile phone). 

In this tutorial we will see how can the author convert the regular whatsapp chats to python code and execute them on whatsapp. This project is a small IDE which runs on whatsapp and can do all the functionalities of python on whatsapp.

In the full tutorial, the author will be creating a bot to run python codes on WhatsApp. It will be kind of creating his own code editor.

Links:

• Full Tutorial - https://www.youtube.com/watch?v=hW3zfd0mIfw
• Python on WhatsApp - https://github.com/pyGuru123/Python-on-WhatsApp
  

Abhinavsingh Proxy.py

Do you need a lightweight proxy server that support HTTP/HTTPS/HTTP2/WebSockets, to facilitate end-to-end testing?

Here's a stripped-down version of lightweight proxy server (written in Python).

 

Summary:

• Fast and scalable and lightweight
  
• Programmable (with plugins)
  
• Real time dashboard
• Supports: HTTP (1.0, 1.1, 2.0), HTTPS and websockets
• IPv4 and IPv6 support
• Basic authentication support

 

Links:

•  https://github.com/abhinavsingh/proxy.py