Formulating Problems

This video discusses the fundamental task of problem-solving and emphasizes the importance of precise problem formulation through the use of Domain Specific Languages (DSLs), contrasting this approach with the inherent vagueness and limitations of natural language.

Understanding and Formulating Problems
#

[00:00:00] The video introduces problem-solving as a fundamental generic task, defining it as the process of moving from a state of a problem not being solved to a state where it is solved or progress is made.
[00:00:42] Proper formulation is key, stressing the importance of understanding the vocabulary and jargon of the problem’s domain, which in computer science is referred to as a Domain Specific Language (DSL).
[00:01:32] Building languages (DSLs) to solve problems is presented as a powerful viewpoint, applicable across various domains. The power lies in representing states, behaviors, and transformations as symbols and using higher abstractions.
[00:02:15] Using the example of making coffee, the speaker illustrates the ambiguity of natural language (“making a cup of coffee”) compared to the necessary precision needed for a machine to solve the problem (“a cup with some coffee in it”).
[00:03:24] Formulating a problem requires defining base abstractions and available resources (e.g., microwave, water, instant coffee, cup).
[00:04:14] Solving a specific kind of problem repeatedly benefits from facilitating the application of a domain (a DSL) rather than just a specific solution.

The Power and Structure of DSLs
#

[00:04:45] A convex optimizer library is given as an example of a DSL, where the vocabulary is based on math (matrix algebra, calculus) allowing users to work with high-level concepts like matrices, constraints, and gradients instead of low-level machine details.
[00:05:45] Using any higher-level programming language is a step towards building a base language/jargon (DSL) for effectively solving problems and expressing classes of solutions.
[00:06:20] The elaborateness (richness of base abstractions), power, expressiveness, and encapsulation of a DSL determine the intricacy and value of the solutions it can express. This is why DSLs are crucial for efficiency and reusability.
[00:07:49] The concept of a meta-DSL or a generic problem solver is discussed, exploring the idea of giving a solver access to base abstractions and an oracle to determine if the problem is solved, allowing it to explore solutions.

Natural Language vs. DSLs and Modern Takes
#

[00:10:02] Natural language is deemed too vague for precise problem formulation, leading to interpretation issues and the need for iterating on implicit assumptions. While convenient for quick fixes, it’s inefficient for problems encountered repeatedly.
[00:11:48] Building DSLs is presented as the most efficient long-term approach to problem-solving, allowing users to solve problems faster over time and improve the DSL itself.
[00:12:26] Natural language forces users to adopt unclear assumptions. The speaker contrasts using natural language (“cut twice and never measure”) with building DSLs (“measure twice… cut once”).
[00:13:30] Modern approaches like LLMs can delegate the implicit conversion of natural language to base assumptions and procedures by training on vast datasets, but this relies on statistical probability rather than true understanding.
[00:15:03] The speaker questions the continued heavy reliance on natural language in engineering domains where powerful abstractions (math, physics, computing) already exist. Natural language is more helpful in vague domains or pseudosciences.

Conclusion and Recommendation
#

[00:17:05] To truly understand and solve problems effectively, particularly in engineering, it is best to understand the DSL of the domain – its vocabulary and jargon.
[00:19:20] Diving into the DSL of a domain helps comprehend and express concepts better. Language is powerful, and understanding its structure allows expressing behaviors, attributes, and transformations formally using symbols (hinting at concepts like symbolic computation and lambda calculus).
[00:20:54] The main takeaway is to not limit oneself to natural language, but to put effort into understanding and building DSLs for better problem comprehension and efficient, long-term solutions.

The video encourages viewers to delve into the specific vocabulary and structured approach of Domain Specific Languages for efficient and profound problem-solving, especially in technical fields, rather than relying on the often ambiguous nature of natural language.