Information Technology Articles

Cheat Sheet / Updated 05-22-2025

The field of SAS and SAS programming has evolved over nearly 50 years, leading to the development of various shorthand techniques. These techniques may not be immediately apparent to new SAS users, but they become clear with learning and practice. Use the tips here to get a head-start and accelerate your initiation to SAS.

Cheat Sheet / Updated 05-12-2025

A geographic information system (GIS) is a software for making maps, analyzing data, and more. This cheat sheet tells you about what you can do with GIS, provides a handy guide to raster-based functions, gives you some key ideas to keep in mind about maps (relating to scales, projections, and datums), and the X, Y, and Z of GIS.

Article / Updated 02-20-2025

Artificial intelligence (AI) offers a lot of promise to companies, but deploying AI can be complex with many considerations and pitfalls. Data is the necessary asset to make AI work, and your organization is probably swimming in it. But how well-managed is your data? The answer hinges on whether your company will be successful in its AI efforts. Data management is the behind-the scenes workhorse that makes AI work. A robust management program allows data to be ingested from everywhere it needs to be, cleaned and transformed to enable AI model training, made easily available to users, and meticulously governed to ensure security, privacy, and compliance. In this article, we’ll explore several ways a data management platform can help with your AI efforts. Although these applications may differ, effective data management is always the necessary first step on which these solutions are built. Proprietary AI As your business scales, the number of receipts, invoices, contracts, and other printed documents scale, too. And when all those documents aren’t digitized, think of the number of hours it will take an employee to catalog it. It’s possible to use AI to automate this process. A proprietary engine scans and processes documents, extracts meaning from them, and outputs the data in a format that’s handy for reports, dashboards, and business intelligence apps. Some benefits of using proprietary AI to scan and process documents are: It can translate multiple languages. A large language model (LLM) can be trained to make sense of any specific document formats that your company may have. Accurate data helps with decision-making. Data can be extracted from third-party platforms, enrich it, validate it, and output it to dashboards accessible throughout the company. Identify issues with customers earlier. Algorithms aggregate and analyze user data, spotlighting any issues with customers early on to prevent customer churn. Or to spotlight when a loyal customer is ready to grow with your company. Retrieval augmented generation Things that work well in a controlled environment with a carefully curated data sample don’t always work in a real-world environment. One such situation is with retrieval augmented generation (RAG), the engine that LLMs rely on to give accurate facts. But if RAG is relying on legacy data that wasn’t prepared adequately, your AI solution is going to underperform. A data management program makes sure the basic, but vitally important, tasks are covered — data is cleaned, engineered, structured, and complete. Some tasks it can do are: Implement meta-intent branching for handling different types of queries. Develop verified quotes and see-it-in-source features for transparency. Monitor and balance token consumption. Improve data quality through semantic data scrubbing. Research and development Traditional research and development methods can be time consuming and expensive. Applying AI to the process can help reduce the cost and release products to the market faster. Reliable products help retain customers, boost the company’s reputation, and grow profit margin. High-quality data is needed to make it all work. A data management program can help with the following tasks: Automate manual processes. Automating helps to lower errors and inefficiencies, and accelerates quantitative research by navigating unstructured data. Business decisions get made faster. Verify and vet output. One system can generate formulas or prototypes for new products; a secondary one can automatically evaluate, compare, and check them for compatibility and other parameters. Enhance new network implementations and diagnostics. AI can create potential scenarios for the design and deployment of new systems. It can create hypotheses to pinpoint problems and suggest solutions. About the Book Wiley has recently published AI Data Management For Dummies, Keboola Special Edition. It includes insights from Snowflake and Capgemini that will help your organization integrate best practices and advanced technologies into your data strategy, what the future of AI development looks like, and more use cases. Download AI Data Management For Dummies, Keboola Special Edition by Andy Mott, Dan O’Riordan, and Rithesh Makkena to open the door to AI success.

Article / Updated 12-09-2024

As you delve deeper into the realm of prompt engineering, you find out that the effectiveness of a prompt can vary greatly depending on the application. Whether you’re using AI for creative writing, data analysis, customer service, or any other specific use, the prompts you use need to be tailored to fit the task at hand. The art in prompt engineering is matching your form of communication to the nature of the task. If you succeed, you’ll unlock the vast potential of AI. For instance, when engaging with AI for creative writing, your prompts should be open-ended and imaginative, encouraging the AI to generate original and diverse ideas. A prompt like “Write a story about a lost civilization discovered by a group of teenagers” sets the stage for a creative narrative. In contrast, data analysis requires prompts that are precise and data-driven. Here, you might need to guide the AI with specific instructions or questions, such as “Analyze the sales data from the last quarter and identify the top-performing products.” You may need to include that data in the prompt if it isn’t already loaded into the training data, retrieval-augmented generation (RAG), system or custom messages, or a specialized GPT. In any case, this type of prompt helps the AI focus on the exact task, ensuring that the output is relevant and actionable. The key to designing effective prompts lies in understanding the domain you’re addressing. Each field has its own set of terminologies, expectations, and objectives. For example, legal prompts require a different structure and language than those used in entertainment or education. It’s essential to incorporate domain-specific knowledge into your prompts to guide the AI in generating the desired output. Following are some examples across various industries that illustrate how prompts can be tailored for domain-specific applications: Legal domain: In the legal industry, precision and formality are paramount. Prompts must be crafted to reflect the meticulous nature of legal language and reasoning. For instance, a prompt for contract analysis might be, “Identify and summarize the obligations and rights of each party as per the contract clauses outlined in Section 2.3 and 4.1.” This prompt is structured to direct the AI to focus on specific sections, reflecting the detailed-oriented nature of legal work. Healthcare domain: In healthcare, prompts must be sensitive to medical terminology and patient privacy. A prompt for medical diagnosis might be, “Given the following anonymized patient symptoms and test results, what are the potential differential diagnoses?” This prompt respects patient confidentiality while leveraging the AI’s capability to process medical data. Education domain: Educational prompts often aim to engage and instruct. A teacher might use a prompt like, “Create a lesson plan that introduces the concept of photosynthesis to 5th graders using interactive activities.” This prompt is designed to generate educational content that is age-appropriate and engaging. Finance domain: In finance, prompts need to be data-driven and analytical. A financial analyst might use a prompt such as, “Analyze the historical price data of XYZ stock over the past year and predict the trend for the next quarter based on the moving average and standard deviation.” This prompt asks the AI to apply specific financial models to real-world data. Marketing domain: Marketing prompts often focus on creativity and audience engagement. A marketing professional could use a prompt like, “Generate a list of catchy headlines for our new eco-friendly product line that will appeal to environmentally conscious consumers.” This prompt encourages the AI to produce creative content that resonates with a target demographic. Software development domain: In software development, prompts can be technical and require understanding of coding languages. A prompt might be, “Debug the following Python code snippet and suggest optimizations for increasing its efficiency.” This prompt is technical, directing the AI to engage with code directly. Customer service domain: For customer service, prompts should be empathetic and solution oriented. A prompt could be, “Draft a response to a customer complaint about a delayed shipment, ensuring to express understanding and offer a compensatory solution.” This prompt guides the AI to handle a delicate situation with care. By understanding the unique requirements and language of each domain, you can craft prompts to effectively guide AI in producing the desired outcomes. It’s not just about giving commands; it’s about framing them in a way that aligns with the goals, terms, and practices of the industry in question. As AI continues to evolve, the ability to engineer precise and effective prompts becomes an increasingly valuable skill across all sectors. 15 tips and tricks for better AI prompting Although GenAI may seem like magic, it takes knowledge and practice to write effective prompts that will generate the content you’re looking for. The following list provides some insider tips and tricks to help you optimize your prompts to get the most out of your interactions with GenAI tools: Know your goal. Decide what you want from the AI — like a simple how-to or a bunch of ideas — before you start asking. Get specific. The clearer you are, the better the AI can help. Ask “How do I bake a beginner's chocolate cake?” instead of just “How do I make a cake?” Keep it simple. Use easy language unless you’re in a special field like law or medicine where using the right terms is necessary. Add context. Give some background if it's a special topic, like tips for small businesses on social media. Play pretend. Tell the AI to act like someone, like a fitness coach, to get answers that fit that role. Try again. If the first answer isn't great, change your question a bit and ask again. Show examples. If you want something creative, show the AI an example to follow, like asking for a poem like one by Robert Frost. Don't overwhelm. Keep your question focused. If it's too packed with info, it gets messy. Mix it up. Try asking in different ways, like with a question or a command, to see what works best. Embrace the multimodal functionality. Multimodal functionality means that the GenAI model you’re working with can accept more than one kind of prompt input. Typically, that means it can accept both text and images in the input. Understand the model’s limitations. GenAI is not infallible and can still produce errors or “hallucinate” responses. Always approach the AI’s output with a critical eye and use it as a starting point rather than the final word on any subject. Leverage the enhanced problem-solving abilities. GenAI’s enhanced problem-solving skills mean that you can tackle more complex prompts. Use this to your advantage when crafting prompts that require a deep dive into a topic. Keep prompts aligned with AI training. For example, remember that GPT-4, like its predecessors, is trained on a vast dataset up to a certain point in time (April 2023 at the time of this writing). It doesn’t know about anything that happened after that date. If you need to reference more recent events or data, provide that context within your prompt. Experiment with different prompt lengths. Short prompts can be useful for quick answers, while longer, more detailed prompts can provide more context and yield more comprehensive responses. Incorporate feedback loops. After receiving a response from your GenAI application, assess its quality and relevance. If it hit — or is close to — the mark, click on the thumbs-up icon. If it’s not quite what you were looking for, provide feedback in your next prompt by clicking on the thumbs-down icon. This iterative process can help refine the AI’s understanding of your requirements and improve the quality of future responses. By keeping these tips in mind and staying informed about the latest developments in the capabilities of various GenAI models and applications, you’ll be able to craft prompts that are not only effective but also responsible and aligned with the AI’s strengths and limitations. How to use prompts to fine-tune the AI model The point of prompt engineering is to carefully compose a prompt that can shape the AI’s learning curve and fine-tune its responses to perfection. In this section, you dive into the art of using prompts to refine the GenAI model, ensuring that it delivers the most accurate and helpful answers possible. In other words, you discover how to use prompts to also teach the model to perform better for you over time. Here are some specific tactics: When you talk to the AI and it gives you answers, tell it if you liked the answer or not. Do this by clicking the thumbs up or thumbs down, or the + or – icons above or below the output. The model will learn how to respond better to you and your prompts over time if you do this consistently. If the AI gives you a weird answer, there's a “do-over” button you can press. It's like asking your friend to explain something again if you didn't get it the first time. Look for “Regenerate Response'’ or some similar wording (term varies among models) near the output. Click on that and you’ll instantly get the AI’s second try! Think of different ways to ask the AI the same or related questions. It's like using magic words to get the best answers. If you're really good at it, you can make a list of prompts that others can use to ask good questions too. Prompt libraries are very helpful to all. It’s smart to look at prompt libraries for ideas when you’re stumped on how or what to prompt. Share your successful prompts. If you find a super good way to ask something, you can share it online (at sites like GitHub) with other prompt engineers and use prompts others have shared there too. Instead of teaching the AI everything from scratch (retraining the model), you can teach it a few more new things through your prompting. Just ask it in different ways to do new things. Over time, it will learn to expand its computations. And with some models, what it learns from your prompts will be stored in its memory. This will improve the outputs it gives you too! Redirect AI biases. If the AI says something that seems mean or unfair, rate it a thumbs down and state why the response was unacceptable in your next prompt. Also, change the way you ask questions going forward to redirect the model away from this tendency. Be transparent and accountable when you work with AI. Tell people why you're asking the AI certain questions and what you hope to get from it. If something goes wrong, try to make it right. It's like being honest about why you borrowed your friend's toy and fixing it if it breaks. Keep learning. The AI world changes a lot, and often. Keep up with new models, features, and tactics, talk to others, and always try to get better at making the AI do increasingly more difficult things. The more you help GenAI learn, the better it gets at helping you! What to do when AI goes wrong When you engage with AI through your prompts, be aware of common pitfalls that can lead to biased or undesirable outcomes. Following are some strategies to avoid these pitfalls, ensuring that your interactions with AI are both effective and ethically sound. Recognize and mitigate biases. Biases in AI can stem from the data it was trained on or the way prompts are structured. For instance, a healthcare algorithm in the United States inadvertently favored white patients over people of color because it used healthcare cost history as a proxy for health needs, which correlated with race. To avoid such biases, carefully consider the variables and language used in your prompts. Ensure they do not inadvertently favor one group over another or perpetuate stereotypes. Question assumptions. Wrong or flawed assumptions can lead to misguided AI behavior. For example, Amazon’s hiring algorithm developed a bias against women because it was trained on resumes predominantly submitted by men. Regularly review the assumptions behind your prompts and be open to challenging and revising them as needed. Avoid overgeneralization. AI can make sweeping generalizations based on limited data. To prevent this, provide diverse and representative examples in your prompts. This helps the AI understand the nuances and variations within the data, leading to more accurate and fair outcomes. Keep your purpose in sight. Losing sight of the purpose of your interaction with AI can result in irrelevant or unhelpful responses. Always align your prompts with the intended goal and avoid being swayed by the AI’s responses into a direction that deviates from your original objective. Diversify information sources. Relying on too narrow a set of information can skew AI responses. Ensure that the data and examples you provide cover a broad spectrum of scenarios and perspectives. This helps the AI develop a well-rounded understanding of the task at hand. For example, if the AI is trained to find causes of helicopter crashes and the only dataset the AI has is of events when helicopters crash, it will deduce that all helicopters crash which in turn will render skewed outputs that could be costly or even dangerous. Add data on flights or events when helicopters did not crash, and you’ll get better outputs because the model has more diverse and more complete information to analyze. Encourage open debate. AI can sometimes truncate debate by providing authoritative-sounding answers. Encourage open-ended prompts that allow for multiple viewpoints and be critical of the AI’s responses. This fosters a more thoughtful and comprehensive exploration of the topic. Be wary of consensus. Defaulting to consensus can be tempting, especially when AI confirms our existing beliefs. However, it’s important to challenge the AI and yourself by considering alternative viewpoints and counterarguments. This helps in uncovering potential blind spots and biases. Check your work. Always review the AI’s responses for accuracy and bias. As with the healthcare algorithm that skewed resources toward white patients, unintended consequences can arise from seemingly neutral variables. Rigorous checks and balances are necessary to ensure the AI’s outputs align with ethical standards.

Video / Updated 11-13-2024

When you’re new to crafting AI prompts, you can easily make mistakes. Using AI tools the right way makes you more productive and efficient. But if you aren’t careful, you may develop bad habits when you’re still learning. We clue you in to 10 mistakes you should avoid from the start in this video and article. Not Spending Enough Time Crafting and Testing Prompts One common mistake when using AI tools is not putting in the effort to carefully craft your prompts. You may be tempted — very tempted — to quickly type out a prompt and get a response back from the AI, but hurried prompts usually produce mediocre results. Taking the time to compose your prompt using clear language will increase your chances of getting the response you want. A poor response spells the need for you to evaluate the prompt to see where you can clarify or improve it. It’s an iterative process, so don’t be surprised if you have to refine your prompt several times. Like any skill, learning to design effective prompts takes practice and patience. The key is to resist the urge to take shortcuts. Make sure to put in the work needed to guide the AI to a great response. Assuming the AI Understands Context or Subtext It’s easy to overestimate the capabilities of AI tools and assume they understand the meaning of language the way humans do. Current AI tools take things literally. They don’t actually understand the context of a conversation. An AI assistant may be trained to identify patterns and connections and is aware of these things as concepts (like norms, emotions, or sarcasm), all of which rely on context, but it struggles to identify them reliably. Humans can read between the lines and understand meaning beyond what’s actually written. An AI interprets instructions and prompts in a very literal sense — it doesn’t understand the meaning behind them. You can’t assume an AI understands concepts it hasn’t been trained for. Asking Overly Broad or Vague Questions When interacting with an AI, avoid overly broad or vague questions. The AI works best when you give it clear, specific prompts. Providing prompts like “Tell me about human history” or “Explain consciousness” is like asking the AI to search the entire internet. The response will probably be unfocused. The AI has no sense of what information is relevant or important so you need to refocus and try again. Good prompts are more direct. You can start with a prompt such as “Summarize this research paper in two paragraphs” or “Write a 500-word article on summer plants that require shade.” The prompt should give the AI boundaries and context to shape its response. Going from broad to increasingly narrow questions also helps. You can start generally asking about a topic and then follow up with focused requests on the specific details. Providing concrete examples guides the AI. The key is to give the AI precise prompts centered directly on the information you want instead of typing a request with a vague, borderless question. Sharp, specific questioning produces the best AI results. Not Checking Outputs for Errors and Biases A common mistake when using AI apps is taking the results at face value without double-checking them. AI systems may reflect bias, or generate text that seems right but has errors. Just because the content came from an AI doesn’t mean it’s necessarily accurate. Reviewing AI responses rather than blindly trusting the technology is critical. Look for instances of bias where specific demographics are negatively characterized or tropes (clichés) are reinforced. Always check facts and figures against other sources. Look for logic that indicates the AI was “confused.” Providing feedback when the AI makes a mistake can further enhance its training. The key is to approach responses skeptically instead of assuming that the AI always generates perfect results. As with any human team member, reviewing their work is essential before using it. Careful oversight of AI tools mitigates risks. Using Offensive, Unethical, or Dangerous Prompts A primary concern when working with AI is that the apps can inadvertently amplify harmful biases if users write offensive, unethical, or dangerous prompts. The AI will generate text for any input, but the response may be that you’re asking for a harmful response and it will not comply. Prompting an AI with inappropriate language or potential discrimination may reinforce biases from the data the model was trained on. If users are cautious when formulating prompts, that can help steer the technology toward more thoughtful responses. AI can be subject to the whims of bad actors. Expecting Too Much Originality or Creativity from the AI One common mistake when using AI apps is expecting too much original thought or creativity. AI tools can generate unique mixes of text, imagery, and other media, but there are limits. As of this writing, AI apps are only capable of remixing existing information and patterns into new combinations. They can’t really create responses that break new ground. An AI has no natural creative flair like human artists or thinkers. Its training data consists only of past and present works. So, although an AI can generate new work, expecting a “masterpiece” is unrealistic. Copying Generated Content Verbatim A big mistake users make when first using AI tools is to take the text and use it verbatim, without any edits or revisions. AI can often produce text that appears to be well written, but the output is more likely to be a bit rough and require a good edit. Mindlessly copying the unedited output can result in unclear and generic work. (Also, plagiarizing or passing the writing off as your own is unethical.) A best practice is to use the suggestions as a starting point that you build upon with your own words and edits to polish the final product. Keep the strong parts and make it into something original. The key is that the AI app should support your work, not replace it. With the right editing and polishing, you can produce something you’ll be proud of. Providing Too Few Examples and Use Cases When you’re training an AI app to handle a new task, a common mistake is to provide too few examples of inputs. Humans can usually extrapolate from a few samples, but AI apps can’t. An AI must be shown examples to grasp the full scope of the case. You need to feed the AI varied use cases to help it generalize effectively. Similarly, limiting prompts to just a couple of instances produces equally poor results because the AI has little indication of the boundaries of the task. Providing diverse examples helps the AI form an understanding about how to respond. Having patience and supplying many examples lets the AI respond appropriately. Not Customizing Prompts for Different Use Cases One common mistake when working with AI tools is attempting to use the same generic prompt to handle all your use cases. Creating a one-size-fits-all prompt is easier, but it will deliver disappointing results. Each use case and application has its own unique goals and information that need to be conveyed, as discussed throughout this book. For example, a prompt for a creative nonfiction story should be designed differently than a prompt for a medical article. An inventory of prompts designed for various use cases allows the AI to adapt quickly to different needs. The key is customization. Building a library of specialized prompts is an investment that pays dividends. Becoming Overly Reliant on AI Tasks Better Suited for Humans Almost everyone is excited about using AI tools to make their job easier. But it’s important to avoid becoming too dependent on them. AI is great for tasks like automation and personalization, but applying ethics and conveying empathy are still human strengths.

Article / Updated 11-06-2024

In the ever-accelerating race of data processing and analytics, your organization’s ability to adapt and evolve its data architecture is crucial. As we enter the next data cycle, marked by the rise of artificial intelligence (AI) and deep learning (DL), the demands on data storage and management are unprecedented. This is where hyperscale network-attached storage (NAS) comes into play, offering a transformative solution for organizations looking to capitalize on the next wave of data-driven opportunities. Ushering in the Next Data Cycle with Hyperscale NAS The next data cycle is characterized by a shift from structured business intelligence (BI) data to a world where unstructured and semi-structured data reign supreme. This data is massive in volume and comes from countless sources, which require high-performance access to drive valuable insights. Hyperscale NAS meets this challenge head-on by merging the simplicity of enterprise NAS with the extreme performance of high-performance computing (HPC) parallel file systems. Hyperscale NAS is essential for your organization for the following reasons: Performance at scale: Hyperscale NAS isn’t bottlenecked by traditional NAS controllers, so linear scalability in performance and capacity are enabled across potentially thousands of nodes. Cost efficiency: Being software-defined, hyperscale NAS allows the use of commodity hardware and avoids vendor lock-in, driving down costs significantly. Metadata excellence: With shared metadata kept out of the data path, hyperscale NAS ensures fast access to data across the global environment, which is critical for AI and DL workloads that require rapid data retrieval and processing. Hyperscale NAS: A Pillar for Modern Data Architectures The architecture of hyperscale NAS is fundamentally different from traditional solutions. It overcomes the limitations of scale-out NAS by providing linear scalability and extreme throughput using commodity infrastructure. This means that as your data grows, your system’s performance and capacity can grow with it, without the need for expensive, specialized hardware. Some of the transformative capabilities of hyperscale NAS include Standards-based integration: The hyperscale NAS client software is built into standard Linux distributions, eliminating the need for proprietary clients. Consistent high performance: The separation of metadata from the data path allows for near-full bandwidth utilization, delivering the speed necessary for demanding applications. High availability: Hyperscale NAS includes distributed architecture, load balancing, tiering, and high-availability features, proving itself in some of the world’s largest AI environments. Making Data Available and Actionable In the era of AI, having access to sufficient compute resources, like graphics processing units (GPUs), is a significant challenge. Hyperscale NAS helps you leverage GPUs wherever they’re available, making data a live, globally shared resource that’s no longer localized or trapped within proprietary storage systems or specific cloud data services. Hyperscale NAS makes data more accessible and actionable through Global data sets: Hyperscale NAS orchestrates data to GPUs in the cloud and GPU-as-a-Service providers, unifying multiple data sources into a single global file system. Intelligent data placement: Leveraging metadata-driven data orchestration, Hyperscale NAS ensures data is where it needs to be, when it needs to be there. Non-disruptive data mobility: Data can be moved between storage systems, sites, and clouds without disrupting access or performance. As your organization gears up for the next data cycle, adopting hyperscale NAS isn’t just a strategic move; it’s an imperative one. It’s a future-proof solution that enables you to keep pace with the exponential growth of data and the complex demands of AI and DL. With hyperscale NAS, you can transform your business, unleash the full potential of your data assets, and ensure that your organization isn’t just ready but thriving in the next data cycle. For more information, download this free e-book: Hyperscale NAS For Dummies, Hammerspace Special Edition.

Article / Updated 10-28-2024

Bayes’ theorem can help you deduce how likely something is to happen in a certain context, based on the general probabilities of the fact itself and the evidence you examine, and combined with the probability of the evidence given the fact. Seldom will a single piece of evidence diminish doubts and provide enough certainty in a prediction to ensure that it will happen. As a true detective, to reach certainty, you have to collect more evidence and make the individual pieces work together in your investigation. Noticing that a person has long hair isn’t enough to determine whether person is female or a male. Adding data about height and weight could help increase confidence. The Naïve Bayes algorithm helps you arrange all the evidence you gather and reach a more solid prediction with a higher likelihood of being correct. Gathered evidence considered singularly couldn’t save you from the risk of predicting incorrectly, but all evidence summed together can reach a more definitive resolution. The following example shows how things work in a Naïve Bayes classification. This is an old, renowned problem, but it represents the kind of capability that you can expect from an AI. The dataset is from the paper “Induction of Decision Trees,” by John Ross Quinlan. Quinlan is a computer scientist who contributed to the development of another machine learning algorithm, decision trees, in a fundamental way, but his example works well with any kind of learning algorithm. The problem requires that the AI guess the best conditions to play tennis given the weather conditions. The set of features described by Quinlan is as follows: Outlook: Sunny, overcast, or rainy Temperature: Cool, mild, or hot Humidity: High or normal Windy: True or false The following table contains the database entries used for the example: Outlook Temperature Humidity Windy PlayTennis Sunny Hot High False No Sunny Hot High True No Overcast Hot High False Yes Rainy Mild High False Yes Rainy Cool Normal False Yes Rainy Cool Normal True No Overcast Cool Normal True Yes Sunny Mild High False No Sunny Cool Normal False Yes Rainy Mild Normal False Yes Sunny Mild Normal True Yes Overcast Mild High True Yes Overcast Hot Normal False Yes Rainy Mild High True No The option of playing tennis depends on the four arguments shown here. The result of this AI learning example is a decision as to whether to play tennis, given the weather conditions (the evidence). Using just the outlook (sunny, overcast, or rainy) won’t be enough, because the temperature and humidity could be too high or the wind might be strong. These arguments represent real conditions that have multiple causes, or causes that are interconnected. The Naïve Bayes algorithm is skilled at guessing correctly when multiple causes exist. The algorithm computes a score, based on the probability of making a particular decision and multiplied by the probabilities of the evidence connected to that decision. For instance, to determine whether to play tennis when the outlook is sunny but the wind is strong, the algorithm computes the score for a positive answer by multiplying the general probability of playing (9 played games out of 14 occurrences) by the probability of the day’s being sunny (2 out of 9 played games) and of having windy conditions when playing tennis (3 out of 9 played games). The same rules apply for the negative case (which has different probabilities for not playing given certain conditions): likelihood of playing: 9/14 * 2/9 * 3/9 = 0.05 likelihood of not playing: 5/14 * 3/5 * 3/5 = 0.13 Because the score for the likelihood is higher, the algorithm decides that it’s safer not to play under such conditions. It computes such likelihood by summing the two scores and dividing both scores by their sum: probability of playing : 0.05 / (0.05 + 0.13) = 0.278 probability of not playing : 0.13 / (0.05 + 0.13) = 0.722 You can further extend Naïve Bayes to represent relationships that are more complex than a series of factors that hint at the likelihood of an outcome using a Bayesian network, which consists of graphs showing how events affect each other. Bayesian graphs have nodes that represent the events and arcs showing which events affect others, accompanied by a table of conditional probabilities that show how the relationship works in terms of probability. The figure shows a famous example of a Bayesian network taken from a 1988 academic paper, “Local computations with probabilities on graphical structures and their application to expert systems,” by Lauritzen, Steffen L. and David J. Spiegelhalter, published by the Journal of the Royal Statistical Society. The depicted network is called Asia. It shows possible patient conditions and what causes what. For instance, if a patient has dyspnea, it could be an effect of tuberculosis, lung cancer, or bronchitis. Knowing whether the patient smokes, has been to Asia, or has anomalous x-ray results (thus giving certainty to certain pieces of evidence, a priori in Bayesian language) helps infer the real (posterior) probabilities of having any of the pathologies in the graph. Bayesian networks, though intuitive, have complex math behind them, and they’re more powerful than a simple Naïve Bayes algorithm because they mimic the world as a sequence of causes and effects based on probability. Bayesian networks are so effective that you can use them to represent any situation. They have varied applications, such as medical diagnoses, the fusing of uncertain data arriving from multiple sensors, economic modeling, and the monitoring of complex systems such as a car. For instance, because driving in highway traffic may involve complex situations with many vehicles, the Analysis of MassIve Data STreams (AMIDST) consortium, in collaboration with the automaker Daimler, devised a Bayesian network that can recognize maneuvers by other vehicles and increase driving safety.

Step by Step / Updated 10-28-2024

Windows usually detects the presence of a network adapter automatically; typically, you don’t have to install device drivers manually for the adapter. When Windows detects a network adapter, Windows automatically creates a network connection and configures it to support basic networking protocols. You may need to change the configuration of a network connection manually, however. The following steps show you how to configure your network adapter on a Windows 10 system:

Cheat Sheet / Updated 10-17-2024

The first public release of ChatGPT ignited the world’s demand for increasingly sophisticated Generative AI (GenAI) models and tools, and the market was quick to deliver. But what’s the use of having so many GenAI tools if you get stuck using them? And make no mistake, everyone gets stuck quite often! This cheat sheet helps you get the very best results by introducing you to advanced (but pretty easy) prompting techniques and giving you useful tips on how to choose models or applications that are right for the task.

Article / Updated 09-24-2024

Both linear and logistic regression see a lot of use in data science but are commonly used for different kinds of problems. You need to know and understand both types of regression to perform a full range of data science tasks. Of the two, logistic regression is harder to understand in many respects because it necessarily uses a more complex equation model. The following information gives you a basic overview of how linear and logistic regression differ. The equation model Any discussion of the difference between linear and logistic regression must start with the underlying equation model. The equation for linear regression is straightforward. y = a + bx You may see this equation in other forms and you may see it called ordinary least squares regression, but the essential concept is always the same. Depending on the source you use, some of the equations used to express logistic regression can become downright terrifying unless you’re a math major. However, the start of this discussion can use one of the simplest views of logistic regression: p = f(a + bx) >p, is equal to the logistic function, f, applied to two model parameters, a and b, and one explanatory variable, x. When you look at this particular model, you see that it really isn’t all that different from the linear regression model, except that you now feed the result of the linear regression through the logistic function to obtain the required curve. The output (dependent variable) is a probability ranging from 0 (not going to happen) to 1 (definitely will happen), or a categorization that says something is either part of the category or not part of the category. (You can also perform multiclass categorization, but focus on the binary response for now.) The best way to view the difference between linear regression output and logistic regression output is to say that the following: Linear regression is continuous. A continuous value can take any value within a specified interval (range) of values. For example, no matter how closely the height of two individuals matches, you can always find someone whose height fits between those two individuals. Examples of continuous values include: Height Weight Waist size Logistic regression is discrete. A discrete value has specific values that it can assume. For example, a hospital can admit only a specific number of patients in a given day. You can’t admit half a patient (at least, not alive). Examples of discrete values include: Number of people at the fair Number of jellybeans in the jar Colors of automobiles produced by a vendor The logistic function Of course, now you need to know about the logistic function. You can find a variety of forms of this function as well, but here’s the easiest one to understand: f(x) = e^x / e^x + 1 You already know about f, which is the logistic function, and x equals the algorithm you want to use, which is a + bx in this case. That leaves e, which is the natural logarithm and has an irrational value of 2.718, for the sake of discussion (check out a better approximation of the whole value). Another way you see this function expressed is f(x) = 1 / (1 + e^-x) Both forms are correct, but the first form is easier to use. Consider a simple problem in which a, the y-intercept, is 0, and ">b, the slope, is 1. The example uses x values from –6 to 6. Consequently, the first f(x) value would look like this when calculated (all values are rounded): (1) e^-6 / (1 + e^-6) (2) 0.00248 / 1 + 0.00248 (3) 0.002474 As you might expect, an xvalue of 0 would result in an f(x) value of 0.5, and an x value of 6 would result in an f(x) value of 0.9975. Obviously, a linear regression would show different results for precisely the same x values. If you calculate and plot all the results from both logistic and linear regression using the following code, you receive a plot like the one below. import matplotlib.pyplot as plt %matplotlib inline from math import exp x_values = range(-6, 7) lin_values = [(0 + 1*x) / 13 for x in range(0, 13)] log_values = [exp(0 + 1*x) / (1 + exp(0 + 1*x)) for x in x_values] plt.plot(x_values, lin_values, 'b-^') plt.plot(x_values, log_values, 'g-*') plt.legend(['Linear', 'Logistic']) plt.show() This example relies on list comprehension to calculate the values because it makes the calculations clearer. The linear regression uses a different numeric range because you must normalize the values to appear in the 0 to 1 range for comparison. This is also why you divide the calculated values by 13. The exp(x) call used for the logistic regression raises e to the power of x, e^x, as needed for the logistic function. The model discussed here is simplified, and some math majors out there are probably throwing a temper tantrum of the most profound proportions right now. The Python or R package you use will actually take care of the math in the background, so really, what you need to know is how the math works at a basic level so that you can understand how to use the packages. This section provides what you need to use the packages. However, if you insist on carrying out the calculations the old way, chalk to chalkboard, you’ll likely need a lot more information. The problems that logistic regression solves You can separate logistic regression into several categories. The first is simple logistic regression, in which you have one dependent variable and one independent variable, much as you see in simple linear regression. However, because of how you calculate the logistic regression, you can expect only two kinds of output: Classification: Decides between two available outcomes, such as male or female, yes or no, or high or low. The outcome is dependent on which side of the line a particular data point falls. Probability: Determines the probability that something is true or false. The values true and false can have specific meanings. For example, you might want to know the probability that a particular apple will be yellow or red based on the presence of yellow and red apples in a bin. Fit the curve As part of understanding the difference between linear and logistic regression, consider this grade prediction problem, which lends itself well to linear regression. In the following code, you see the effect of trying to use logistic regression with that data: x1 = range(0,9) y1 = (0.25, 0.33, 0.41, 0.53, 0.59, 0.70, 0.78, 0.86, 0.98) plt.scatter(x1, y1, c='r') lin_values = [0.242 + 0.0933*x for x in x1] log_values = [exp(0.242 + .9033*x) / (1 + exp(0.242 + .9033*x)) for x in range(-4, 5)] plt.plot(x1, lin_values, 'b-^') plt.plot(x1, log_values, 'g-*') plt.legend(['Linear', 'Logistic', 'Org Data']) plt.show() The example has undergone a few changes to make it easier to see precisely what is happening. It relies on the same data that was converted from questions answered correctly on the exam to a percentage. If you have 100 questions and you answer 25 of them correctly, you have answered 25 percent (0.25) of them correctly. The values are normalized to produce values between 0 and 1 percent. As you can see from the image above, the linear regression follows the data points closely. The logistic regression doesn’t. However, logistic regression often is the correct choice when the data points naturally follow the logistic curve, which happens far more often than you might think. You must use the technique that fits your data best, which means using linear regression in this case. A pass/fail example An essential point to remember is that logistic regression works best for probability and classification. Consider that points on an exam ultimately predict passing or failing the course. If you get a certain percentage of the answers correct, you pass, but you fail otherwise. The following code considers the same data used for the example above, but converts it to a pass/fail list. When a student gets at least 70 percent of the questions correct, success is assured. y2 = [0 if x < 0.70 else 1 for x in y1] plt.scatter(x1, y2, c='r') lin_values = [0.242 + 0.0933*x for x in x1] log_values = [exp(0.242 + .9033*x) / (1 + exp(0.242 + .9033*x)) for x in range(-4, 5)] plt.plot(x1, lin_values, 'b-^') plt.plot(x1, log_values, 'g-*') plt.legend(['Linear', 'Logistic', 'Org Data']) plt.show() This is an example of how you can use list comprehensions in Python to obtain a required dataset or data transformation. The list comprehension for y2 starts with the continuous data in y1 and turns it into discrete data. Note that the example uses precisely the same equations as before. All that has changed is the manner in which you view the data, as you can see below. Because of the change in the data, linear regression is no longer the option to choose. Instead, you use logistic regression to fit the data. Take into account that this example really hasn’t done any sort of analysis to optimize the results. The logistic regression fits the data even better if you do so.