This repository is a visual cheatsheet on the main topics in Backend-development. All the material is divided into topics and subtopics. The structure of the material consists of three parts:
-**Visual part** - various images/tables/cheatsheets for better understanding (may not be available). All pictures and tables are made from scratch, specifically for this repository.
-**Summary** - A very brief summary with a list of key terms and concepts. The terms are hyperlinked to the appropriate section on Wikipedia or a similar reference resource.
-**References to sources** - resources where you may find complete information on a particular issue. If possible, the most authoritative sources are indicated, or those that provide information in as simple and comprehensible language as possible.
> 🛠 The repository is under active development, so it is constantly updated and supplemented
* [Monolithic and microservice architecture](#monolithic-and-microservice-architecture)
* [Horizontal and vertical scaling](#horizontal-and-vertical-scaling)
</details>
[Additional and similar resources](#additional-and-similar-resources)
## Network & Internet
[Internet](https://en.wikipedia.org/wiki/Internet) is a worldwide system that connects computer networks from around the world into a single network for storing/transferring information. The Internet was originally developed for the military. But soon it began to be implemented in universities, and then it could be used by private companies, which began to organize networks of providers that provide Internet access services to ordinary citizens. By early 2020, the number of Internet users exceeded 4.5 billion.
Your computer has never been directly connected to the Internet. Because it can only see its local network to which other devices are connected via wired ([Ethernet](https://en.wikipedia.org/wiki/Ethernet)) or wirelessly (Wi-Fi, Bluetooth). To communicate with the Internet, you have a special minicomputer in your local network - [router](<https://en.wikipedia.org/wiki/Router_(computing)>). It then connects you to [Internet Service Provider](https://en.wikipedia.org/wiki/Internet_service_provider) which in turn connects to other higher-level providers. Thus, your message, transits through the network of several ISPs before reaching the destination network.
The Internet is just a long wire to which a small number of [Tier 1 providers](https://en.wikipedia.org/wiki/Tier_1_network) are directly connected. The ISPs below that are just renting access.
[Domain Names](https://en.wikipedia.org/wiki/Domain_name) are human-readable addresses of web servers available on the Internet. They consist of parts (levels) separated from each other by a dot. Each of these parts provides specific information about the domain name. For example country, service name, localization, etc.
- Who owns domain names
> [The ICANN Corporation](https://en.wikipedia.org/wiki/ICANN) is the founder of the distributed domain registration system. It gives accreditations to companies that want to sell domains. In this way a competitive domain market is formed.
- How to buy a domain name
> A domain name cannot be bought forever. It is leased for a certain period of time. It is better to buy domains from [accredited registrars](https://www.icann.org/en/accredited-registrars?filter-letter=a&sort-direction=asc&sort-param=name&page=1) (you can find them in almost any country).
<details>
<summary>🔗 <b>References</b></summary>
1. 📄 [**What is a Domain Name?** – MDN](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_is_a_domain_name)
2. 📺 [**A Beginners Guide to How Domain Names Work!** – YouTube](https://youtu.be/Y4cRx19nhJk)
[IP address](https://en.wikipedia.org/wiki/IP_address) is a unique numeric address that is used to recognize a particular device on the network.
- Levels of visibility
> - External and publicly accessible IP address that belongs to your ISP and is used to access the Internet by hundreds of other users.
> - The IP address of your router in your ISP's local network, the same IP address from which you access the Internet.
> - The IP address of your computer in the local (home) network created by the router, to which you can connect your devices. Typically, it looks like 192.168.XXX.XXX.
> - The internal IP address of the computer, inaccessible from the outside and used only for communication between the running processes. It is the same for everyone - 127.0.0.1 or just _localhost_.
> One device (computer) can run many applications that use the network. In order to correctly recognize where and which data coming over the network should be delivered (to which of the applications) a special numerical number - a port is used. That is, each running process on a computer which uses a network connection has its own personal port.
- [IPv4](https://en.wikipedia.org/wiki/IPv4)
> Version 4 of the IP protocol. It was developed in 1981 and limits the address space to about 4.3 billion (2^32) possible unique addresses.
- [IPv6](https://en.wikipedia.org/wiki/IPv6)
> Over time, the allocation of address space began to happen at a much faster rate, forcing the creation of a new version of the IP protocol to store more addresses. IPv6 is capable of issuing 2^128 (is huge number) unique addresses.
[DNS (Domain Name System)](https://en.wikipedia.org/wiki/DNS) is a decentralized Internet address naming system that allows you to create human-readable alphabetic names (domain names) corresponding to the numeric [IP addresses](#ip-address) used by computers.
<details>
<summary>🔗 <b>References</b></summary>
1. 📄 [**What is DNS? Domain Name System explained** – freeCodeCamp](https://www.freecodecamp.org/news/what-is-dns/)
2. 📺 [**DNS (Domain Name System) explained. Types of Domain Name Servers** – YouTube](https://youtu.be/JkEYOt08-rU)
3. 📺 [**DNS as Fast As Possible** – YouTube](https://youtu.be/Rck3BALhI5c)
Modern [web applications](https://en.wikipedia.org/wiki/Web_application) consist of two parts: [Frontend and Backend](https://en.wikipedia.org/wiki/Frontend_and_backend). Thus implementing a [client-server model](https://en.wikipedia.org/wiki/Client%E2%80%93server_model).
The tasks of the Frontend are:
- Implementation of the user interface (appearance of the application)
> A special markup language [HTML](https://en.wikipedia.org/wiki/HTML) is used to create web pages. <br> > [CSS](https://en.wikipedia.org/wiki/CSS) style language is used to style fonts, layout of content, etc. <br> > [JavaScript](https://en.wikipedia.org/wiki/JavaScript) programming language is used to add dynamics and interactivity. <br>
> As a rule, these tools are rarely used in their pure form, as so-called [frameworks](https://2020.stateofjs.com/en-US/technologies/front-end-frameworks/) and [preprocessors](https://www.freecodecamp.org/news/css-preprocessors/) exist for more convenient and faster development. <br>
- Creating functionality for generating requests to the server
> These are usually different types of input forms that can be conveniently interacted with.
- Receives data from the server and then processes it for output to the client
Tasks of the Backend:
- Handling client requests
> Checking for permissions and access, all sorts of validations, etc.
- Implementing business logic
> A wide range of tasks can be implied here: working with databases, information processing, computation, etc. This is, so to speak, the heart of the Backend world. This is where all the important and interesting stuff happens.
- Generating a response and sending it to the client
[Browser](https://en.wikipedia.org/wiki/Web_browser) is a client which can be used to send requests to a server for files which can then be used to render web pages. In simple terms, a browser can be thought of as a program for viewing HTML files, which can also search for and download them from the Internet.
The use of VPNs and Proxy is quite common in recent years. With the help of these technologies, users can get basic anonymity when surfing the web, as well as bypass various regional blockages.
> A technology that allows you to become a member of a private network (similar to your local network), where requests from all participants go through a single public IP address. This allows you to blend in with the general mass of requests from other participants. <br>
>
> - Simple procedure for connection and use. <br>
> - Reliable traffic encryption. <br>
> - There is no guarantee of 100% anonymity, because the owner of the network knows the IP-addresses of all participants. <br>
> - VPNs are useless for dealing with multi-accounts and some programs because all accounts operating from the same VPN are easily detected and blocked. <br>
> - Free VPNs tend to be heavily loaded, resulting in unstable performance and slow download speeds. <br>
> A proxy is a special server on the network that acts as an intermediary between you and the destination server you intend to reach. When you are connected to a proxy server all your requests will be performed on behalf of that server, that is, your IP address and location will be substituted. <br>
>
> - The ability to use an individual IP address, which allows you to work with multi-accounts. <br>
> - Stability of the connection due to the absence of high loads. <br>
> - Connection via proxy is provided in the operating system and browser, so no additional software is required. <br>
> - There are proxy varieties that provide a high level of anonymity. <br>
> - The unreliability of free solutions, because the proxy server can see and control everything you do on the Internet. <br>
[Hosting](https://en.wikipedia.org/wiki/Web_hosting_service) is a special [service provided](https://en.wikipedia.org/wiki/Internet_hosting_service) by hosting providers, which allows you to rent space on a server (which is connected to the Internet around the clock), where your data and files can be stored. There are different options for hosting, where you can use not only the disk space of the server, but also the CPU power to run your network applications.
> At this level, bits (ones/zeros) are encoded into physical signals (current, light, radio waves) and transmitted further by wire ([Ethernet](https://en.wikipedia.org/wiki/Ethernet)) or wirelessly ([Wi-Fi](https://en.wikipedia.org/wiki/Wi-Fi)).
- [Data link layer](https://en.wikipedia.org/wiki/Data_link_layer)
> Physical signals from layer 1 are decoded back into ones and zeros, errors and defects are corrected, and the sender and receiver [MAC addresses](https://en.wikipedia.org/wiki/MAC_address) are extracted.
> The layer responsible for data transfer. There are two important protocols: <br>
>
> - [TCP](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) is a protocol that ensures reliable data transmission. TCP guarantees data delivery and preserves the order of the messages. This has an impact on the transmission speed. This protocol is used where data loss is unacceptable, such as when sending mail or loading web pages. <br>
> - [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol) is a simple protocol with fast data transfer. It does not use mechanisms to guarantee the delivery and ordering of data. It is used e.g. in online games where partial packet loss is not crucial, but the speed of data transfer is much more important. Also, requests to DNS servers are made through UDP protocol.
> Responsible for opening and closing communications (sessions) between two devices. Ensures that the session stays open long enough to transfer all necessary data, and then closes quickly to avoid wasting resources.
> Transmission, encryption/decryption and data compression. This is where data that comes in the form of zeros and ones are converted into desired formats (PNG, MP3, PDF, etc.)
[HTTP (HyperText Transport Protocol)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) is the most important protocol on the Internet. It is used to transfer data of any format. The protocol itself works according to a simple principle: request -> response.
- [Structure of HTTP messages](https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages)
> [GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET) - data retrieval request <br> [POST](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST) - request with data to create a new record <br> [PUT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PUT) - request with data to change existing record <br> [DELETE](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/DELETE) - deletion request <br> Others: [HEAD](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD), [CONNECT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT), [OPTIONS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS), [TRACE](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/TRACE), [PATCH](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PATCH). <br>
- [Response status codes](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status)
> Each response from the server has a special numeric code that characterizes the state of the sent request. These codes are divided into 5 main classes:
> Because the HTTP protocol does not allow you to save any information about the status of previous requests/responses, you need to use cookies. Cookies allow the server to store various information on the client side, which the client can then send back to the server. In particular, cookies can be used for authorization or to save various settings/configurations.
> A special header that allows you to recognize and eliminate certain types of web application vulnerabilities.
- [HTTP/1.0 vs HTTP/1.1 vs HTTP/2](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Persistent_connections)
> The main innovation in version 1.1 is the permanent connection mode, which allows you to send several requests per connection. In version 2, the protocol became binary, with the ability to transmit data from multiple streams on the same channel.
Compared to the [OSI model](https://github.com/cheatsnake/backend-cheats/blob/master/README_ENG.md#osi-network-model), the [TCP/IP](https://en.wikipedia.org/wiki/Internet_protocol_suite) stack has a simpler architecture. It is widely used and was first used as the basis for the creation of a global network, and then to describe the workings of the Internet.
1. 📄 [**What is the TCP/IP Model? Layers and Protocols Explained** – freeCodeCamp](https://www.freecodecamp.org/news/what-is-tcp-ip-layers-and-protocols-explained/)
2. 📺 [**What is TCP/IP?** – YouTube](https://youtu.be/PpsEaqJV_A0)
The quality of networks, much less the Internet, is far from ideal. This is due to the complex and dispersed network structure in different devices. Therefore, on the functioning of the network affects a huge number of factors. For example: the stability of the connection between the client device and its router, the quality of service of the provider, the power and performance of the server, the physical distance between the client and the server, etc.
> Not all packets traveling over the network can reach their destination. This happens most often when using wireless networks or due to [network congestion](https://en.wikipedia.org/wiki/Network_congestion).
- [Round Trip Time (RTT)](https://en.wikipedia.org/wiki/Round-trip_delay)
> The time it takes for the data packet to reach its destination + the time to respond that the packet was received successfully.
> A procedure that allows you to trace to which nodes, with which IP addresses, a packet you send before it reaches its destination. Tracing can be used to identify computer network related problems and to examine/analyze the network.
> The most important PC component to which all other elements are connected.
>
> - [Chipset](https://en.wikipedia.org/wiki/Chipset) - set of electronic components that responsible for the communication of all motherboard components.
> - [CPU socket](https://en.wikipedia.org/wiki/CPU_socket) - socket for mounting the processor.
> - [VRM (Voltage Regulator Module)](https://en.wikipedia.org/wiki/Voltage_regulator_module) – module that converts the incoming voltage (usually 12V) to a lower voltage to run the processor, integrated graphics, memory, etc.
> - Slots for RAM.
> - Expansion slots [PCI-Express](https://en.wikipedia.org/wiki/PCI_Express) - designed for connection of video cards, external network/sound cards.
> - Slots [М.2](https://en.wikipedia.org/wiki/M.2) / [SATA](https://en.wikipedia.org/wiki/SATA) - designed to connect hard disks and SSDs.
> The most important device that executes instructions (programme code). Processors only work with 1 and 0, so all programmes are ultimately a set of binary code.
>
> - [Registers](https://en.wikipedia.org/wiki/Processor_register) - the fastest memory in a PC, has an extremely small capacity, is built into the processor and is designed to temporarily store the data being processed.
> - [Cache](https://en.wikipedia.org/wiki/CPU_cache) - slightly less fast memory, which is also built into the processor and is used to store a copy of data from frequently used cells in the main memory.
> - Processors can have different [architectures](https://en.wikipedia.org/wiki/Processor_design). Currently, the most common are the [x86](https://en.wikipedia.org/wiki/X86-64) architecture (desktop and laptop computers) and [ARM](https://en.wikipedia.org/wiki/ARM_architecture_family) (mobile devices as well as the latest Apple computers).
> Fast, low capacity memory (4-16GB) designed to temporarily store program code, as well as input, output and intermediate data processed by the processor.
> A separate card that translates and processes data into images for display on a monitor. This device is also called a discrete graphics card. Usually needed for those who do 3D modelling or play games. <br> > [Built-in graphics card](https://en.wikipedia.org/wiki/Graphics_processing_unit#Integrated_graphics_processing_unit) is a graphics card built into the processor. It is suitable for daily work.
[Operating system (OS)](https://en.wikipedia.org/wiki/Operating_system) is a comprehensive software system designed to manage a computer's resources. With operating systems, people do not have to deal directly with the processor, RAM or other parts of the PC.
OS can be thought of as an abstraction layer that manages the hardware of a computer, thereby providing a simple and convenient environment for user software to run.
- Main features
> - RAM management (space allocation for individual programms)
> - Loading programms into RAM and their execution
> - Execution of requests from user's programms (inputting and outputting data, starting and stopping other programms, freeing up memory or allocating additional memory, etc.)
> - Interaction with input and output devices (mouse, keyboard, monitor, etc.)
> - Interaction with storage media (HDDs and SSDs)
> - Providing a user's interface (console shell or graphical interface)
> - Logging of software errors (saving logs)
- Additional functions (may not be available in all OSs)
> - Organise [multitasking](https://en.wikipedia.org/wiki/Computer_multitasking) (simultaneous execution of several programms)
> - Delimiting access to resources for each process
> The central part of the operating system which is used most intensively. The kernel is constantly in memory, while other parts of the OS are loaded into and unloaded from memory as needed.
> The system software that prepares the environment for the OS to run (puts the hardware in the right state, prepares the memory, loads the OS kernel there and transfers control to it (the kernel).
1. 📄 [**What is an OS? Operating System Definition for Beginners** – freeCodeCamp](https://www.freecodecamp.org/news/what-is-an-os-operating-system-definition-for-beginners/)
2. 📄 [**Windows vs MacOS vs Linux – Operating System Handbook** – freeCodeCamp](https://www.freecodecamp.org/news/an-introduction-to-operating-systems/)
> A kind of container in which all the resources needed to run a program are stored. As a rule, the process consists of:
>
> - Executable program code <br>
> - Input and output data <br>
> - [Call stack](https://en.wikipedia.org/wiki/Call_stack) (order of instructions for execution) <br>
> - [Heap](https://en.wikipedia.org/wiki/Memory_management#Manual_memory_management) (a structure for storing intermediate data created during the process) <br>
> An entity in which sequences of program actions (procedures) are executed. Threads are within a process and use the same address space. There can be multiple threads in a single process, allowing multiple tasks to be performed. These tasks, thanks to threads, can exchange data, use shared data or the results of other tasks.
> The ability to perform multiple tasks, but using a single processor core. This is achieved by dividing tasks into separate blocks of commands which are executed in turn, but switching between these blocks is so fast that for users it seems as if these processes are running simultaneously.
1. 📄 [**Concurrency, parallelism, and the many threads of Santa Claus** – freeCodeCamp](https://www.freecodecamp.org/news/concurrency-parallelism-and-the-many-threads-of-santa-claus/)
2. 📺 [**Concurrency vs Parallelism** – YouTube](https://youtu.be/Y1pgpn2gOSg)
3. 📺 [**Concurrency is not Parallelism by Rob Pike** – YouTube](https://youtu.be/oV9rvDllKEg)
A mechanism which allows to exchange data between threads of one or different processes. Processes can be run on the same computer or on different computers connected by a network. [Inter-process communication](https://en.wikipedia.org/wiki/Inter-process_communication) can be done in different ways.
> The easiest way to exchange data. One process writes data to a certain file, another process reads the same file and thus receives data from the first process.
> In particular, IP addresses and ports are used to communicate between computers using the TCP/IP protocol stack. This pair defines a socket (_socket_ corresponding to the address and port).
Operating systems based on [Linux kernel](https://en.wikipedia.org/wiki/Linux_kernel) are the standard in the world of server development, since most servers run on such operating systems. Using Linux on servers is profitable because it is free.
There are a huge number of Linux distributions (preinstalled software bundles) to suit all tastes. One of the most popular is [Ubuntu](https://en.wikipedia.org/wiki/Ubuntu). This is where you can start your dive into server development.
[Install Ubuntu](https://ubuntu.com/download/desktop) on a separate PC or laptop. If this is not possible, you can use a special program [Virtual Box](https://www.virtualbox.org/wiki/Downloads) where you can [run other OS]() on top of the main OS. You can also run [Docker](https://www.docker.com/products/docker-desktop) [Ubuntu image container](https://hub.docker.com/_/ubuntu) (Docker is a [separate topic](#docker) that is exists in this repository).
- ### Working with the terminal
[Terminal](https://en.wikipedia.org/wiki/Computer_terminal) is a program that uses special text commands to control your computer. Generally, servers do not have graphical interfaces, so you will definitely need terminal skills.
- Basic commands for navigating the file system
```bash
ls # list directory contents
cd <path> # go to specified directory
cd .. # move to a higher level (to the parent directory)
touch <file> # create a file
cat > <file> # enter text into the file (overwrite)
cat >> <file> # enter text at the end of the file (append)
cat/more/less <file> # to view the file contents
head/tail <file> # view the first/last lines of a file
pwd # print path to current directory
mkdir <name> # create a directory
rmdir <name> # delete a directory
cp <file><path> # copy a file or directory
mv <file><path># moving or renaming
rm <file> # deleting a file or directory
find <string># file system search
du <file># output file or directory size
```
- Commands for help information
```bash
man <command> # allows you to view a manual for any command
apropos <string> # search for a command with a description that has a specified word
man -k <string> # similar to the command above
whatis <command> # a brief description of the command
```
- Super user rights
> Analogue to running as administrator in Windows
```bash
sudo <command> # executes a command with superuser privileges
```
- Text editor
> Study any in order to read and edit files freely through the terminal.
> The easiest – [nano](https://en.wikipedia.org/wiki/GNU_nano).
> The most advanced – [Vim](<https://en.wikipedia.org/wiki/Vim_(text_editor)>).
<details>
<summary>🔗 <b>References</b></summary>
1. 📄 [**31 Linux Commands Every Ubuntu User Should Know**](https://itsfoss.com/essential-ubuntu-commands/)
2. 📄 [**The Linux Command Handbook** – freeCodeCamp](https://www.freecodecamp.org/news/the-linux-commands-handbook/)
3. 📺 [**The 50 Most Popular Linux & Terminal Commands** – YouTube](https://youtu.be/ZtqBQ68cfJc)
The Package Manager is a utility that allows you to install/update software packages from the terminal.
Linux distributions can be divided into several groups, depending on which package manager they use: [apt](<https://en.wikipedia.org/wiki/APT_(software)>) (in [Debian](https://en.wikipedia.org/wiki/Debian) based distributions), [RPM](https://en.wikipedia.org/wiki/RPM_Package_Manager) (the [Red Hat](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux) package management system) and [Pacman](https://en.wikipedia.org/wiki/Arch_Linux#Pacman) (the package manager in [Arch-like distributions](https://en.wikipedia.org/wiki/Arch_Linux))
Ubuntu is based on Debian, so it uses apt (advanced packaging tool) package manager.
- Basic Commands
```bash
apt install <package> # install the package
apt remove <package> # remove the package, but keep the configuration
apt purge <package> # remove the package along with the configuration
apt update # update information about new versions of packages
apt upgrade # update the packages installed in the system
apt list --installed # list of packages installed on the system
apt list --upgradable # list of packages that need to be updated
apt search <package> # searching for packages by name on the network
3. 📄 [**sudo apt-get update vs upgrade – What is the Difference?** – freeCodeCamp](https://www.freecodecamp.org/news/sudo-apt-get-update-vs-upgrade-what-is-the-difference)
You can use scripts to automate the sequential input of any number of commands. In [Bash](<https://en.wikipedia.org/wiki/Bash_(Unix_shell)>) you can create different conditions (branching), loops, timers, etc. to perform all kinds of actions related to console input.
- [Basics of Bash Scripts](./files/linux/bash-scripts-cheatsheet.md)
> The most basic and frequently used features such as: variables, I/O, loops, conditions, etc.
- Practice
> Solve challenges on sites like [HackerRank](https://www.hackerrank.com/domains/shell) and [Codewars](https://www.codewars.com/join?language=shell).
> Start using Bash to automate routine activities on your computer. If you're already a programmer, create scripts to easily build your project, to install settings, and so on.
> It will point out possible mistakes and teach you best practices for writing really good scripts.
- Additional resources
> Repositories such as [awesome bash](https://github.com/awesome-lists/awesome-bash) and [awesome shell](https://github.com/alebcay/awesome-shell) have entire collections of useful resources and tools to help you develop even more skills with Bash and the terminal in general.
Linux-based operating systems are multi-user. This means that several people can run many different applications at the same time on the same computer. For the Linux system to be able to "recognize" a user, he must be logged in and therefore each user must have a unique name and a secret password.
- Working with users
```bash
useradd <name> [flags] # create a new user
passwd <name> # set a password for the user
usermod <name> [flags] # edit a user
usermod -L <name> # block a user
usermod -U <name> # unblock a user
userdel <name> [flags] # delete a user
```
- Working with groups
```bash
groupadd <group> [flags] # create a group
groupmod <group> [flags] # edit group
groupdel <group> [flags] # delete group
usermod -a -G <groups><user> # add a user to groups
gpasswd --delete <user><groups> # remove a user from groups
```
- System files
```bash
/etc/passwd # a file containing basic information about users
/etc/shadow # a file containing encrypted passwords
/etc/group # a file containing basic information about groups
/etc/gshadow # a file containing encrypted group passwords
```
<details>
<summary>🔗 <b>References</b></summary>
1. 📄 [**Managing Users, Groups and Permissions in Linux**](https://omarrrz-lounge.hashnode.dev/managing-users-groups-and-permissions-in-linux)
2. 📄 [**Linux User Groups Explained** – freeCodeCamp](https://www.freecodecamp.org/news/linux-user-groups-explained-how-to-add-a-new-group-a-new-group-member-and-change-groups/)
3. 📺 [**Linux Users and Groups** – YouTube](https://youtu.be/b-9j2jiCOEA)
In Linux, it is possible to share privileges between users, limit access to unwanted files or features, control available actions for services, and much more. In Linux, there are only three kinds of rights - read, write and execute - and three categories of users to which they can be applied - file owner, file group and everyone else.
- Basic commands for working with rights
```bash
chown <user><file> # changes the owner and/or group for the specified files
chmod <rights><file> # changes access rights to files and directories
chgrp <group><file> # allows users to change groups
```
- Extended rights [SUID and GUID](https://en.wikipedia.org/wiki/Setuid), [sticky bit](https://en.wikipedia.org/wiki/Sticky_bit)
- [ACL (Access control list)](https://en.wikipedia.org/wiki/Access-control_list)
> An advanced subsystem for managing access rights.
Linux processes can be described as containers in which all information about the state of a running program is stored. If a program hangs and you need to restore it, then you need the skills to manage the processes.
- Basic Commands
```bash
ps # display a snapshot of the processes of all users
top # real-time task manager
<command>& # running the process in the background, (without occupying the console)
jobs # list of processes running in the background
fg <PID> # return the process back to the active mode by its number
bg <PID> # start a stopped process in the background
kill <PID> # terminate the process by PID
killall <programm> # terminate all processes related to one program
1. 📄 [**How to Show Process Tree in Linux**](https://linuxhandbook.com/show-process-tree/)
2. 📄 [**How To Use ps, kill, and nice to Manage Processes in Linux** – Digital Ocean](https://www.digitalocean.com/community/tutorials/how-to-use-ps-kill-and-nice-to-manage-processes-in-linux)
[SSH]() allows remote access to another computer's terminal. In the case of a personal computer, this may be needed to solve an urgent problem, and in the case of a server, it is generally the primary method of connection.
- Basic commands
```bash
apt install openssh-server # installing SSH (out of the box almost everywhere)
service ssh start # start SSH
service ssh stop # stop SSH
ssh -p <port> user@remote_host # connecting to a remote PC via SSH
ssh-keygen -t rsa # RSA key generation for passwordless login
ssh-copy-id -i ~/.ssh/id_rsa user@remote_host # copying a key to a remote machine
Schedulers allow you to flexibly manage the delayed running of commands and scripts. Linux has a built-in [cron](https://en.wikipedia.org/wiki/Cron) scheduler that can be used to easily perform necessary actions at certain intervals.
- Main commands
```bash
crontab -e # edit the crontab file of the current user
crontab -l # output the contents of the current schedule file
crontab -r # deleting the current schedule file
```
- Config files
```bash
/etc/crontab # base config
/etc/cron.d/ # crontab files used to manage the entire system
[Log files]() are special text files that contain all information about the operation of a computer, program, or user. They are especially useful when bugs and errors occur in the operation of a program or server. It is recommended to periodically review log files, even if nothing suspicious happens.
- Main log files
```bash
/var/log/syslog или /var/log/messages # information about the kernel,
# various services detected, devices, network interfaces, etc.
/var/log/auth.log или /var/log/secure # user authorization information
/var/log/faillog # failed login attempts
/var/log/dmesg # information about device drivers
/var/log/boot.log # operating system boot information
/var/log/cron # cron task scheduler report
```
- [lnav utility](https://lnav.org/)
> Designed for easy viewing of log files (highlighting, reading different formats, searching, etc.)
- Log rotation with [logrotate](https://github.com/logrotate/logrotate)
> Allows you to configure automatic deletion (cleaning) of log files so as not to clog memory.
> Occur due to erroneous actions of the user. Often associated with typos, lack of rights, incorrectly specified options, etc.
- Driver problems
> All free Linux drivers are built right into its kernel. Therefore, everything should work "out of the box" after installing the system (problems may occur with brand new hardware which has just been released on the market). Drivers whose source code is closed are considered proprietary and are not included in the kernel but are installed manually (like Nvidia graphics drivers).
- Problems with kernel
> [Kernel panic]() can occur due to an error when mounting the root file system.
> This is best helped by the skill of reading the logs to find problems (`dmesg` command).
- [Segmentation fault]()
> Occurs when a process accesses invalid memory locations.
[Numeral system](https://en.wikipedia.org/wiki/Numeral_system) is a set of symbols and rules for denoting numbers. In computer science, it is customary to distinguish four main number systems: binary, octal, decimal, and hexadecimal. It is connected, first of all, with their use in various branches of programming.
> The most important system for computing technology. Its use is justified by the fact that the logic of the processor is based on only two states (on/off, open/closed, high/low, true/false, yes/no, high/low).
> A system that is easy to understand for most people.
- [Hexadecimal]()
> The letters A, B, C, D, E, F are additionally used for recording. It is widely used in low-level programming and computer documentation because the minimum addressable memory unit is an 8-bit byte, the values of which are conveniently written in two hexadecimal digits.
[Logical connective](https://en.wikipedia.org/wiki/Logical_connective) are widely used in programming to check various conditions. The result of a logical expression is always _truth_ or _false_.
[Data structures](https://en.wikipedia.org/wiki/Data_structure) are containers in which data is stored according to certain rules. Depending on these rules, the data structure will be effective in some tasks and ineffective in others. Therefore, it is necessary to understand when and where to use this or that structure.
> A data structure where all elements, in addition to the data, contain references to the next and/or previous element. There are 3 varieties:
>
> - A [singly linked list](https://en.wikipedia.org/wiki/Linked_list#Singly_linked_list) is a list where each element stores a link to the next element only (one direction).
> - A [doubly linked list](https://en.wikipedia.org/wiki/Doubly_linked_list) is a list where the items contain links to both the next item and the previous one (two directions).
> - A [circular linked list](https://en.wikipedia.org/wiki/Linked_list#Circular_linked_list) is a kind of bilaterally linked list, where the last element of the ring list contains a pointer to the first and the first to the last.
> In other words, it is an associative array. Here, each of the elements is accessed with a corresponding key value, which is calculated using [hash function](https://en.wikipedia.org/wiki/Hash_function) according to a certain algorithm.
> Similar to the tree, but in the heap, the items with the largest key is the root node (max-heap). But it may be the other way around, then it is a min heap.
1. 📺 [**CS50 2022 - Lecture about Data Structures** – YouTube](https://youtu.be/X8h4dq9Hzq8)
2. 📺 [**Data Structures Easy to Advanced Course** – YouTube](https://youtu.be/RBSGKlAvoiM)
3. 📄 [**Free courses to learn data structures and algorithms in depth** – freeCodeCamp](https://www.freecodecamp.org/news/these-are-the-best-free-courses-to-learn-data-structures-and-algorithms-in-depth-4d52f0d6b35a/)
4. 📄 [**Data Structures: collection of topics** – GeeksForGeeks](https://www.geeksforgeeks.org/data-structures/)
[Algorithms](https://de.wikipedia.org/wiki/Algorithmus) refer to sets of sequential instructions (steps) that lead to the solution of a given problem. Throughout human history, a huge number of algorithms have been invented to solve certain problems in the most efficient way. Accordingly, the correct choice of algorithms in programming will allow you to create the fastest and most resource-intensive solutions.
> There is a very good book on algorithms for beginners – [Grokking algorithms](https://edu.anarcho-copy.org/Algorithm/grokking-algorithms-illustrated-programmers-curious.pdf). You can start [learning a programming language](#programming-language) in parallel with it.
> When a function can call itself and so on to infinity. On the one hand, recursion-based solutions look very elegant, but on the other hand, this approach quickly leads to stack overflow and is recommended to be avoided.
In the world of programming there is a special unit of measure **Big O** (or O-notation). It describes how the complexity of an algorithm increases with the amount of input data. **Big O** estimates how many actions (steps/iterations) it takes to execute the algorithm, while always showing the worst case scenario.
Different file formats can be used to store and transfer data over the network. Text files are human-readable, so they are used for configuration files, for example. But transferring data in text formats over the network is not always rational, because they weigh more than their corresponding binary files.
Computers work only with numbers, or more precisely, only with 0 and 1. It is already clear how to convert numbers from different number systems to binary. But you can't do that with text. That's why special tables called [encodings](https://en.wikipedia.org/wiki/Character_encoding) were invented, in which text characters are assigned numeric equivalents.
- [ASCII (American standard code for information interchange)](https://en.wikipedia.org/wiki/ASCII)
> The simplest encoding created specifically for the American alphabet. Consists of 128 characters.
> This is an international character table that, in addition to the English alphabet, contains the alphabets of almost all countries. It can hold more than a million different characters (the table is currently incomplete).
- [UTF-8](https://en.wikipedia.org/wiki/UTF-8)
> Unicode is a variable-length encoding that can be used to represent any unicode character.
- [UTF-16](https://en.wikipedia.org/wiki/UTF-16)
> Its main difference from UTF-8 is that its structural unit is not one but two bytes. That is, in UTF-16 any Unicode character can be encoded by either two or four bytes.
At this stage you have to choose one programming language to study. There is plenty of information on various languages in the Internet (books, courses, thematic sites, etc.), so you should have no problem finding information.
> Below is a list of specific languages that [personally, in my opinion](https://github.com/cheatsnake) are good for backend development (⚠️ may not agree with the opinions of others, including those more competent in this matter).
> No less popular and practically the only language for full-fledged Web-development. Thanks to the platform [Node.js](https://en.wikipedia.org/wiki/Node.js) last few years is gaining popularity in the field of backend development as well.
> A language created internally by Google. It was created specifically for high-load server development. Minimalistic syntax, high performance and rich standard library.
> A kind of modern version of [Java](<https://en.wikipedia.org/wiki/Java_(programming_language)>). Simpler and more concise syntax, better type-safety, built-in tools for multithreading. One of the best choices for Android development.
Find a good book or online tutorial in English at [this repository](https://github.com/EbookFoundation/free-programming-books/blob/main/books/free-programming-books-langs.md). There is a large collection for different languages and frameworks.
Look for a special [awesome repository](https://github.com/sindresorhus/awesome#programming-languages) - a resource that contains a huge number of useful links to materials for your language (libraries, cheat sheets, blogs and other various resources).
There are many programming languages. They are all created for a reason. Some languages may be very specific and used only for certain purposes. Also, different languages may use different approaches to writing programs. They may even run differently on a computer. In general, there are many different [classifications](https://en.wikipedia.org/wiki/Category:Programming_language_classification), which would be useful to understand.
> As close to machine code as possible, complex to write, but as productive as possible. As a rule, it provides access to all of the computer's resources.
> They have a fairly high level of abstraction, which makes them easy to write and easy to use. As a rule, they are safer because they do not provide access to all of the computer's resources.
- [Compiled, interpreted and embedded languages](https://en.wikipedia.org/wiki/Programming_language#Implementation)
> A powerful tool for working with strings. Be sure to familiarize yourself with it in your language, at least on a basic level.
- Package Manager
> Sooner or later, there will be a desire to use third-party libraries.
After mastering the minimal base for writing the simplest programs, there is not much point in continuing to learn without having specific goals (without practice, everything will be forgotten). You need to think of/find something that you would like to create yourself (a game, a chatbot, a website, a mobile/desktop application, whatever). For inspiration, check out these repositories: [Build your own x](https://github.com/codecrafters-io/build-your-own-x) and [Project based learning](https://github.com/practical-tutorials/project-based-learning).
At this point, the most productive part of learning begins: You just look for all kinds of information to implement your project. Your best friends are Google, YouTube, and Stack Overflow.
> Creation of endpoints (URLs) which will call the appropriate handler on the server when accessed.
- Processing requests
> As a rule, HTTP handlers have a special object which receives all information about user request (headers, method, request body, full url with parameters, etc.)
- Processing responses
> Sending an appropriate message to a received request (HTTP status and code, response body, headers, etc.)
- Error handling
> You should always consider cases where the user could send invalid data, the database failed to execute the operation, or an unexpected error occurred in the application, so that the server does not crash but responds with an error message.
- Sending requests
> Often, within one application, you will need to access another application over the network. That's why it's important to be able to send HTTP requests using the built-in features of the language.
Computers today have processors with several physical and virtual cores, and if we take into account server machines, their number can reach up to hundreds. All of these available resources would be good to use to the fullest, for maximum application performance. That is why modern server development cannot do without implementing [multithreading](<https://en.wikipedia.org/wiki/Multithreading_(computer_architecture)>) and [paralleling](https://en.wikipedia.org/wiki/Parallel_computing).
- [Race conditions & data races](https://en.wikipedia.org/wiki/Race_condition)
> The main problems that arise when using multithreading.
> Lightweight code execution threads organized on top of operating system threads. They can exist as separate libraries or be already built into the kernel.
> Operations that are performed completely, or not performed at all.
- Lockouts
> Using [semaphores](<https://en.wikipedia.org/wiki/Semaphore_(programming)>) and [mutexes](<https://en.wikipedia.org/wiki/Lock_(computer_science)>) to synchronize data.
> A process that has made high-level languages very popular - it allows the programmer not to worry about memory allocation and freeing. Be sure to familiarize yourself with the subtleties of its operation in your own language.
During these long years that programming has existed, a huge amount of code, programs and entire systems have been written. And as a consequence, there have been all sorts of problems in the development of all this. First of all they were related to scaling, support, and the entry threshold for new developers. Clever people, of course, did not sit still and started to solve these problems, thus creating so-called patterns/principles/approaches for writing high-quality code.
By learning programming best practices, you will not only make things better for yourself, but also for others, because other developers will be working with your code.
[Databases (DB)](https://en.wikipedia.org/wiki/Database) – a set of data that are organized according to certain rules (for example, a library is a database for books).
[Database management system (DBMS)](https://en.wikipedia.org/wiki/Database#Database_management_system) is a software that allows you to create a database and manipulate it conveniently (perform various operations on the data). An example of a DBMS is a librarian. He can easily and efficiently work with the books in the library: give out requested books, take them back, add new ones, etc.
Databases can differ significantly from each other and therefore have different areas of application. To understand what database is suitable for this or that task, it is necessary to understand the classification.
> These are repositories where data is organized as a set of tables (with rows and columns). Interactions between data are organized on the basis of links between these tables. This type of database provides fast and efficient access to structured information.
> Here data is represented as objects with a set of attributes and methods. Suitable for cases where you need high-performance processing of data with a complex structure.
> Composed of several parts located on different computers (servers). Such databases may completely exclude information duplication, or completely duplicate it in each distributed copy (for example, as [Blockchain](https://en.wikipedia.org/wiki/Blockchain)).
- [NoSQL](https://en.wikipedia.org/wiki/NoSQL)
> Stores and processes unstructured or weakly structured data. This type of database is subdivided into subtypes:
1. 📄 [**Comparing database types: how database types evolved to meet different needs**](https://www.prisma.io/dataguide/intro/comparing-database-types)
2. 📄 [**SQL vs NoSQL Database – A Complete Comparison**](https://backendless.com/sql-vs-nosql-database-a-complete-comparison/)
The most popular relational databases: [MySQL](https://en.wikipedia.org/wiki/MySQL), [PostgreSQL](https://en.wikipedia.org/wiki/PostgreSQL), [MariaDB](https://en.wikipedia.org/wiki/MariaDB), [Oracle](https://en.wikipedia.org/wiki/Oracle_Database). A special language [SQL (Structured Query Language)](https://postgrespro.com/docs/postgresql/14/sql) is used to work with these databases. It is quite simple and intuitive.
> Sequences of commands that must be executed completely, or not executed at all.
- Command `START TRANSACTION`
- Commands `COMMIT` and `ROLLBACK`
- Working with a programming language
> To do this, you need to install a special driver for your language. For more convenient operation, there are ORM libraries that allow you to execute SQL queries as if you were simply calling methods on the object.
- [Optimization and performance](https://postgrespro.ru/docs/postgresql/14/performance-tips)
[MongoDB](https://en.wikipedia.org/wiki/MongoDB) is a document-oriented database (which is a classic example of [NoSQL](https://en.wikipedia.org/wiki/NoSQL) databases) that does not require a table schema description. Uses JSON-like documents and database schema.
> `MULTI` — start recording commands for the transaction. <br> `EXEC` — execute the recorded commands. <br> `DISCARD` — delete all recorded commands. <br> `WATCH` — command that provides execution only if other clients have not changed the value of the variable. Otherwise EXEC will not execute the written commands.
[ACID](https://en.wikipedia.org/wiki/ACID) is an acronym consisting of the names of the four main properties that guarantee the reliability of transactions in the database.
[Database design](https://en.wikipedia.org/wiki/Database_design) is a very important topic that is often overlooked. A well-designed database will ensure long-term scalability and ease of data maintenance. There are several basic steps in database design:
> The most abstract vision of the entire system is a description of the basic entities (tables) and the relationships between them without focusing on a particular database management system.
- Logic schema
> Work with data models of the future database, the designation of necessary fields with the indication of types, the definition of relationships and relationships with the indication of the primary and external keys.
- Real implementation
> This is the direct work with the DBMS, the organization of specific tables, the implementation of relationships, the creation of indexes, and so on.
[API (Application Programming Interface)](https://en.wikipedia.org/wiki/API) an interface which describes a certain set of rules by which different programs (applications, bots, websites...) can interact with each other. With API calls you can execute certain functions of a program without knowing how it works.
When developing server applications, different API formats can be used, depending on the tasks and requirements.
[REST (Representational State Transfer)](https://ru.wikipedia.org/wiki/REST) an architectural approach that describes a set of rules for how a programmer organizes the writing of server application code so that all systems can easily exchange data and the application can be easily scaled. When building a REST API, HTTP protocol methods are widely used.
Basic rules for writing a good REST API:
- Each URL-endpoint must be a noun
> To perform different operations (get/create/update/delete), this endpoint must implement handlers for the corresponding HTTP methods (GET/POST/PUT/DELETE).
- Use of plural names
> For example, the endpoint to get one record by id looks like this: `/task/42`, and to get all records like this: `/tasks`.
- Using versioning
> Detailed instructions are described [here](https://www.freecodecamp.org/news/how-to-version-a-rest-api/).
- Sending the appropriate HTTP response codes
> The most commonly used: [200](https://developer.mozilla.org/en/docs/Web/HTTP/Status/200), [201](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/201), [204](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/204), [304](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/304), [400](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400), [401](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401), [403](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403), [404](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404), [405](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/405), [410](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/410), [415](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/415), [422](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/422), [429](https://developer.mozilla.org/ru/docs/Web/HTTP/Status/429).
[GraphQL](https://graphql.org/) is a query language that describes how to request data and is mainly used by the client to download data from the server. It has three main features:
- Allows the customer to specify exactly what data they want, thereby reducing traffic consumption from unnecessary data.
- Facilitates the aggregation of data from multiple sources.
- Uses a type system to describe the data.
Content to learn:
- [Queries and Mutations](https://graphql.org/learn/queries/)
- [Schemas and Types](https://graphql.org/learn/schema/)
[WebSockets](https://en.wikipedia.org/wiki/WebSocket) is an advanced technology that allows you to open a persistent bidirectional network connection between the client and the server. With its API you can send a message to the server and receive a response without making an HTTP request, thereby implementing real-time communication.
The basic idea is that you do not need to send requests to the server for new information. When the connection is established, the server itself will send a new batch of data to connected clients as soon as that data is available. Web sockets are widely used to create chat rooms, online games, trading applications, etc.
- Opening a web socket
> Sending an HTTP request with a specific set of headers: `Connection: Upgrade`, `Upgrade: websocket`, `Sec-WebSocket-Key`, `Sec-WebSocket-Version`.
[RPC (remote procedure call)](https://en.wikipedia.org/wiki/Remote_procedure_call) it is simply a function call on the server with a set of defined arguments, which responds with the result of this function call.
The main RPC protocols:
- [SOAP](https://en.wikipedia.org/wiki/SOAP)
> A protocol that uses the XML language. It was developed in 1998. Due to the complexity of XML and high traffic consumption, it is not recommended for use.
[WebRTC](https://en.wikipedia.org/wiki/WebRTC) an open-source project for streaming data (video, audio) in a browser. WebRTC operation is based on [peer to peer connection](https://en.wikipedia.org/wiki/Peer-to-peer), however, there are implementations that allow you to organize complex group sessions. For example, the video-calling service [Google Meet](https://en.wikipedia.org/wiki/Google_Meet) makes extensive use of WebRTC.
[Git](https://en.wikipedia.org/wiki/Git) a special system for managing the history of changes to the source code. Any changes that are made to Git can be saved, allowing you to rollback (revert) to a previously saved copy of the project. Git is currently the standard for development.
- Basic commands
```bash
git init # initialize Git in the current folder
git add [file] # add a file to Git
git add . # add all the files in the folder to Git
git reset [file] # cancel the addition of the specified file
git reset # cancel the addition of all files
git commit -m "your message" # create a commit (save)
git status # shows the status of added files
git push # send current commits to a remote repository
git pull # load changes from a remote repository
git clone [link] # clone the specified repository to your PC
```
- Working with branches
> Branching allows you to deviate from the main branch of development and continue to work independently.
```bash
git branch # show a list of current threads
git branch [name] # create a new branch from the current commit
git checkout [name] # create a new branch from the current commit
git merge [name] # merge the specified branch into the current branch
git branch -d [name] # delete the specified branch
```
- Cancel commits
```bash
git revert HEAD --no-edit # create a new commit that overrides the changes of the previous one
git revert [hash] --no-edit # the same action, but with the specified commit
```
- Log history
```bash
git log [branch] # show the commits of the specified branch
git log -3 # show the last 3 commits of the current branch
git log [file] # show the commit history of the specified file
[Docker](<https://en.wikipedia.org/wiki/Docker_(software)>) a special program that allows you to run isolated sandboxes (containers) with different preinstalled environments (be it a specific operating system, a database, etc.). [Containerization](https://en.wikipedia.org/wiki/OS-level_virtualization) technology, that Docker provides is similar to virtual machines, but unlike virtual machines, containers use the host OS kernel, which requires far fewer resources.
- Docker image
> A special fixed template that contains a description of the environment to run the application (OS, source code, libraries, environment variables, configuration files, etc.). The images can be downloaded from [official site](https://hub.docker.com/search?type=image) and used to create your own.
- Docker container
> An isolated environment created from an image. It is essentially a running process on a computer which internally contains the environment described in the image.
- Console commands
```bash
docker pull [image_name] # Download the image
docker images # List of available images
docker run [image_id] # Running a container based on the selected image
# Some flags for the run command:
-d # Starting with a return to the console
--name [name] # Name the container
--rm # Remove the container after stopping
-p [local_port][port_iside_container] # Port forwarding
docker build [path_to_Dockerfile] # Creating an image based on a Dockerfile
docker ps # List of running containers
docker ps -a # List of all containers
docker stop [id/container_name] # Stop the container
docker start [id/container_name] # Start an existing container
docker attach [id/container_name] # Connect to the container console
docker logs [id/container_name] # Output the container logs
docker rm [id/container_name] # Delete container
docker container prune # Delete all containers
docker rmi [image_id] # Delete image
```
- Instructions for Dockerfile
> Dockerfile is a file with a set of instructions and arguments for creating images.
```bash
FROM [image_name] # Setting a base image
WORKDIR [path] # Setting the root directory inside the container
RUN [command] # A command that runs only when the image is initialized
CMD ["command"] # The command that runs every time you start the container
ENV KEY="VALUE" # Setting Environment Variables
ARG KEY=VALUE # Setting variables to pass to Docker during image building
ENTRYPOINT ["command"] # The command that runs when the container is running
EXPOSE port/protocol # Indicates the need to open a port
VOLUME ["path"] # Creates a mount point for working with persistent storage
```
- Docker-compose
> A tool for defining and running multi-container Docker applications. It allows you to define the services that make up your application in a single file, and then start and stop all of the services with a single command. In a sense, it is a Dockerfile on maximal.
When creating a server application, it is necessary to test its performance. This can be done in different ways. One of the easiest is to use the console utility [curl](https://en.wikipedia.org/wiki/CURL). But this is good for very simple applications. Much more efficient is to use special software for testing, which have a user-friendly interface and all the necessary functionality to create collections of queries.
- [Postman](https://www.postman.com/)
> A very popular and feature-rich program. It definitely has everything you might need and more: from the trivial creation of collections to raising mock-servers. The basic functionality of the application is free of charge.
- [Insomnia](https://insomnia.rest/)
> Not as popular, but a very nice tool. The interface in Insomnia, minimalist and clear. It has less functionality, but everything you need: collections, variables, work with GraphQL, gRPC, WebSocket, etc. It is possible to install third-party plugins.
A [web server](https://en.wikipedia.org/wiki/Web_server) is a program designed to handle incoming HTTP requests. In addition, it can keep error logs (logs), perform authentication and authorization, store rules for file processing, etc.
- What is it for?
> - Not all languages can have a built-in web server (e.g. PHP). Therefore, to run web applications written in such languages, a third-party one is needed.
> - A single server (virtual or dedicated) can run several applications, but only one external IP address. A configured web server solves this problem and can redirect incoming requests to the right applications.
- Popular web servers
> - [Nginx](https://en.wikipedia.org/wiki/Nginx) – the most popular at the moment.
> - [Apache](https://en.wikipedia.org/wiki/Apache_HTTP_Server) – also popular, but already giving up its position.
> - [Caddy](<https://en.wikipedia.org/wiki/Caddy_(web_server)>) – a fairly young web server with great potential.
When creating a large-scale backend system, the problem of communication between a large number of microservices may arise. In order not to complicate existing services (establish a reliable communication system, distribute the load, provide for various errors, etc.) you can use a separate service, which is called a [message broker](https://en.wikipedia.org/wiki/Message_broker) (or message queue).
The broker takes the responsibility of creating a reliable and fault-tolerant system of communication between services (performs balancing, guarantees delivery, monitors recipients, maintains logs, buffering, etc.)
A message is an ordinary HTTP request/response with data of a certain format.
> When a site uses a POST request to perform a transaction, the attacker can forge a form, such as in an email, and send it to the victim. The victim, who is an authorized user interacting with this email, can then unknowingly send a request to the site with the data that the attacker has set.
> The principle is based on the fact that an invisible layer is placed on top of the visible web page, in which the page the intruder wants is loaded, while the control (button, link) needed to perform the desired action is combined with the visible link or button the user is expected to click on.
- [Denial of Service (DoS attack)](https://en.wikipedia.org/wiki/Denial-of-service_attack)
> A hacker attack that overloads the server running the web application by sending a huge number of requests.
> A type of attack in which an attacker gets into the chain between two (or more) communicating parties to intercept a conversation or data transmission.
- Incorrect security configuration
> Using default configuration settings can be dangerous because it is common knowledge. For example, a common vulnerability is that network administrators leave the default logins and passwords _admin:admin_.
<details>
<summary>🔗 <b>References</b></summary>
1. 📺 [**7 Security Risks and Hacking Stories for Web Developers** – YouTube](https://youtu.be/4YOpILi9Oxs)
2. 📄 [**Top 10 Web Application Security Risks**](https://owasp.org/www-project-top-ten/)
Often your applications may use various tokens (e.g. to access a third-party paid API), logins and passwords (to connect to a database), various secret keys for signatures and so on. All this data should not be known and available to outsiders, so you can't leave them in the program code in any case. To solve this problem, there are environment variables.
> Variables are passed to the program using command line arguments. To do the same with the `.env` file, you need to use a special library for your language.
- Storage and transfer `.env` files
> Learn how to upload `.env` files to the hosting services and remember that such files cannot be commited to remote repositories, so do not forget to add them to exceptions via the `.gitignore` file.
Cryptographic algorithms based on [hash functions](https://en.wikipedia.org/wiki/Hash_function) are widely used for network security.
- Hashing
> The process of converting an array of information (from a single letter to an entire literary work) into a unique short string of characters (called hash), which is unique to that array of information. Moreover, if you change even one character in this information array, the new hash will differ dramatically. <br>
> Hashing is an irreversible process, that is, the resulting hash cannot be recovered from the original data.
> Cases where hashing different sets of information results in the same hash.
- [Salt (in cryptography)](<https://en.wikipedia.org/wiki/Salt_(cryptography)>)
> A random string of data, which is added to the input data before hashing, to calculate the hash. This is necessary to make brute-force hacking more difficult.
Popular encryption algorithms:
- [SHA family (Secure Hash Algorithm)](https://en.wikipedia.org/wiki/Secure_Hash_Algorithms)
> [SHA-256](https://en.wikipedia.org/wiki/SHA-2) is the most popular encryption algorithm. It is used, for example, in [Bitcoin](https://en.wikipedia.org/wiki/Bitcoin).
- MD family (Message Digest)
> The most popular algorithm of the family is [MD5](https://en.wikipedia.org/wiki/MD5). It is now considered very vulnerable to collisions (there are even collision generators for MD5).
- [BLAKE](<https://en.wikipedia.org/wiki/BLAKE_(hash_function)>) family
- [RIPEMD](https://en.wikipedia.org/wiki/RIPEMD) family
[Authentication](https://en.wikipedia.org/wiki/Authentication) is a procedure that is usually performed by comparing the password entered by the user with the password stored in the database.
Also, this often includes [identification](<https://en.wikipedia.org/wiki/Identification_(information)>) - a procedure for identifying the user by his unique identifier (usually a regular login or email). This is needed to know exactly which user is being authenticated.
[Authorization](https://en.wikipedia.org/wiki/Authorization) - the procedure of granting access rights to a certain user to perform certain operations. For example, ordinary users of the online store can view products and add them to cart. But only administrators can add new products or delete existing ones.
> The simplest authentication scheme where the username and password of the user are passed in the [Authorization](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization) header in unencrypted (base64-encoded) form. It is relatively secure when using HTTPS.
> Authorization protocol, which allows you to register in various applications using popular services (Google, Facebook, GitHub, etc.)
- [OpenID](https://en.wikipedia.org/wiki/OpenID)
> An open standard that allows you to create a single account for authenticating to multiple unrelated services.
- [JWT (Json Web Token)](https://en.wikipedia.org/wiki/JSON_Web_Token)
> An authentication standard based on access tokens. Tokens are created by the server, signed with a secret key and transmitted to the client, who then uses the token to verify his identity.
[SSL (Secure Socket Layer)](https://en.wikipedia.org/wiki/Transport_Layer_Security#SSL_1.0,_2.0,_and_3.0) and [TLS (Transport Layer Security)](https://en.wikipedia.org/wiki/Transport_Layer_Security) are cryptographic protocols that allow secure transmission of data between two computers on a network. These protocols work essentially the same and there are no differences. SSL is considered obsolete, although it is still used to support older devices.
Testing is the process of assessing that all parts of the program behave as expected of them. Covering the product with the proper amount of testing, allows you to quickly check later to see if anything in the application is broken after adding new or changing old functionality.
The simplest kind of tests. As a rule, about 70-80% of all tests are exactly [unit-tests](https://en.wikipedia.org/wiki/Unit_testing). "Unit" means that not the whole system is tested, but small and separate parts of it (functions, methods, components, etc.) in isolation from others. All dependent external environment is usually covered by [mocks](https://en.wikipedia.org/wiki/Mock_object).
- What are the benefits of unit tests?
> To give you an example, let's imagine a car. Its "units" are the engine, brakes, dashboard, etc. You can check them individually before assembly and, if necessary, replace or repair them. But you can assemble the car without having tested the units, and it will not go. You will have to disassemble everything and check every detail.
- What do I need to start writing unit tests?
> As a rule, the means of the standard language library are enough to write quality tests. But for more convenient and faster writing of tests, it is better to use third-party tools. For example:
>
> - For Python it uses [pytest](https://docs.pytest.org), although the standard [unittest](https://docs.python.org/3/library/unittest.html) is enough to start with.
> - For JavaScript/TypeScript, the best choices are [Jest](https://jestjs.io/).
> - For Go – [testify](https://github.com/stretchr/testify).
> - [And so on...](https://github.com/atinfo/awesome-test-automation#awesome-test-automation)
1. 📺 [**Software Testing Explained in 100 Seconds** – YouTube](https://youtu.be/u6QfIXgjwGQ)
2. 📄 [**How to write your first Unit Test** – medium](https://medium.com/geekculture/how-to-write-your-first-unit-test-in-multiple-programming-languages-6d158d362b3d)
3. 📺 [**How To Write Unit Tests For Existing Python Code** – YouTube](https://youtu.be/ULxMQ57engo)
4. 📺 [**Learn How to Test your JavaScript Application** – YouTube](https://youtu.be/ajiAl5UNzBU)
5. 📺 [**GoLang Unit Testing and Mock Testing Tutorial** – YouTube](https://youtu.be/XQzTUa9LPU8)
