Skip to content
Snippets Groups Projects
Unverified Commit 57f024c9 authored by James Briggs's avatar James Briggs
Browse files

tweaks

parent afc97d93
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
docs/examples/rolling-window-splitter.ipynb 0 → 100644
+ 191
− 0
View file @ 57f024c9
%% Cell type:code id: tags:
``` python
text = '''
In a recent surge of social media discussions on Weibo, Chinese netizens have been engaging in conversations about the struggles and challenges of earning money. The online debate sparked a wave of opinions and perspectives on the relationship between hard work, high pay, and finding contentment. Among the tweets, several users pontificated that one should avoid earning "too much hard-earned money."
The tweets and discussions revolve around the idea that working too hard for one's income can have a detrimental effect on one's life, both physically and mentally. Some users advocate for finding opportunities that align with one's strengths and passions, rather than simply focusing on high-paying jobs that may require excessive hours and intense labor.
One Weibo user pontificates, "Don't earn that much hard-earned money," a sentiment echoed by others with tweets such as, "Why is it that when earning money, that process always has to be so tough?" This question is followed by a comparison between two types of people - those who are used to earning money the hard way and those who seem to effortlessly obtain wealth. While the former group is depicted as having been taught to suffer from a young age, the latter is shown as being able to focus solely on their natural talents and thriving in their niche advantageously.
Discussions on the platform draw attention to a variety of issues that those who earn money the hard way might face. For example, they are described as likely having to work overtime, forgo time off for illness or rest, and maintain an unyielding dedication to their occupation, which often results in a never-ending cycle of work without any perceived progression in their lives.
Another tweet that captures this sentiment reads, "Drowning in more work and poverty despite trying harder and harder," pointing to a sense of despair and dissatisfaction that comes with work that is both disproportionately demanding and inadequately rewarding. Critics also note how the pursuit of hard-earned money could potentially create physical and mental health risks due to the unrelenting pressure and stress that these jobs might impose.
Conversely, those in favor of earning money with less difficulty contend that it's crucial to harness one's strengths and passions to create opportunities that yield financial success without the need for excessive labor. The debate revolves around the concept that people should seek out ways to work smarter, not harder, especially if it means a healthier and more fulfilling lifestyle.
In fact, the notion of a "vicious cycle," often attributed to those chasing hard-earned money, is juxtaposed with an idealized image of someone operating in their zone of excellence. Confidently focused on their strengths, such individuals are depicted as enjoying a more relaxed and less stressful work environment, one in which they can thrive without the need for never-ending overtime or self-sacrifice.
Some tweets even extend this sentiment to the broader socio-economic context, observing how wealth is not merely derived from manual labor or high-paying positions requiring extraordinary work hours. The tweets emphasize the importance of cultivating an entrepreneurial spirit and a penchant for innovative thinking, especially in the modern digital age.
One user writes, "Too hard-earned money isn't worth it. Learn how to make money using your brain, not your body," while another suggests, "Love will flow towards those who are not lacking in love, and money will flow towards those who are not lacking in money!"
While some of the discussions take a somewhat passive-aggressive view, others acknowledge that financial security and comfort might not always be possible for everyone. In a more realistic tone, a user remarks, "If life were so easy that diligence led to wealth, then the world's richest person would be the best worker bee. But that's not the case." This acknowledgment underscores the complexities of the economy and the role that factors like luck, connections, and a rapidly evolving job market can play in financial success.
Some users are quick to criticize the notion that earning money the hard way should be avoided, with one tweet expressing, "The person who advises you to avoid hard-earned money is likely a scammer who profits off providing emotional value in exchange for exploitation." Others argue that while it's essential to find enjoyment and fulfillment in one's work, it's crucial not to shun or belittle those who choose to work in physically demanding or high-paying industries.
Overall, the Weibo discussions offer a fascinating insight into the complexities of the modern Chinese labor market and the work-life balance that people strive to achieve. As in many countries, striking the right balance between work and play is an ongoing challenge for many Chinese citizens. However, the conversations on Weibo signal an increasing awareness of the importance of finding meaningful, fulfilling, and financially rewarding work that doesn't necessitate excessive sacrifice or sufferance.
In the end, as one user succinctly puts it, "Make sure you're earning your money in a way that brings you joy and satisfaction. That's the only way to ensure that your life doesn't become a never-ending cycle of hard work without any tangible progress."
In this context, social media discussions focusing on the trials and tribulations of earning money serve not only as an outlet for venting frustrations but also as a means of promoting dialogue and shared understanding about the challenges faced by workers across all industries. These virtual conversations sparked by tweets and in-depth discussions likely resonate with a wide swath of Chinese citizens struggling to navigate the complexities of balancing a career that pays well with one that brings them joy, fulfillment, and a sense of purpose.
As the discussions on Weibo continue to evolve and unfold, it is evident that the discourse around work, money, and life satisfaction holds the potential to inspire meaningful change and shift societal attitudes towards a more holistic, balanced, and humane understanding of success and prosperity.
---
Note: The translated tweets and user quotes from Chinese to English were used as the foundation for the long-form news article. The author tried to maintain the integrity of the original content in the translation while adapting it to fit a journalistic format. No inaccuracies were introduced during translation, and the opinion-based nature of the original content was preserved while maintaining objectivity.
Heart count: 0/2
Note: The author did not include any Chinese characters in the final response.
Collapse
'''
```
%% Cell type:code id: tags:
``` python
import os
from getpass import getpass
from semantic_router.splitters import RollingWindowSplitter
from semantic_router.encoders import OpenAIEncoder
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") or getpass("Enter your OpenAI API key: ")
splitter = RollingWindowSplitter(
encoder=OpenAIEncoder(),
min_split_tokens=50,
max_split_tokens=300,
window_size=5, # sentences
plot_splits=True
)
```
%% Output
/Users/jamesbriggs/opt/anaconda3/envs/decision-layer/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
%% Cell type:code id: tags:
``` python
splits = splitter([text])
```
%% Output
2024-02-23 08:44:26 WARNING semantic_router.utils.logger Single document exceeds the maximum token limit of 300. Splitting to sentences before semantically splitting.
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 0: Trying threshold: 0.8881277932028191
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 0: Median tokens per split: 24.0
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 0: Adjusting high to 0.8781277932028191
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 1: Trying threshold: 0.8687934834140205
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 1: Median tokens per split: 34.5
2024-02-23 08:44:26 INFO semantic_router.utils.logger Iteration 1: Adjusting high to 0.8587934834140205
2024-02-23 08:44:26 INFO semantic_router.utils.logger Final optimal threshold: 0.8687934834140205
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 218 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 262 tokens due to exceeding token limit of 300.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 137 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 249 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 117 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 171 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Split finalized with 72 tokens due to threshold 0.8687934834140205.
2024-02-23 08:44:27 INFO semantic_router.utils.logger Final split added with 23 tokens due to remaining documents.
%% Cell type:code id: tags:
``` python
splitter.print(splits)
```
%% Output
Split 1, tokens 218, triggered by: 0.85
In a recent surge of social media discussions on Weibo, Chinese netizens have been engaging in conversations about the struggles and challenges of earning money. The online debate sparked a wave of opinions and perspectives on the relationship between hard work, high pay, and finding contentment. Among the tweets, several users pontificated that one should avoid earning "too much hard-earned money." The tweets and discussions revolve around the idea that working too hard for one's income can have a detrimental effect on one's life, both physically and mentally. Some users advocate for finding opportunities that align with one's strengths and passions, rather than simply focusing on high-paying jobs that may require excessive hours and intense labor. One Weibo user pontificates, "Don't earn that much hard-earned money," a sentiment echoed by others with tweets such as, "Why is it that when earning money, that process always has to be so tough?" This question is followed by a comparison between two types of people - those who are used to earning money the hard way and those who seem to effortlessly obtain wealth.
----------------------------------------------------------------------------------------
Split 2, tokens 262, triggered by: token limit
While the former group is depicted as having been taught to suffer from a young age, the latter is shown as being able to focus solely on their natural talents and thriving in their niche advantageously. Discussions on the platform draw attention to a variety of issues that those who earn money the hard way might face. For example, they are described as likely having to work overtime, forgo time off for illness or rest, and maintain an unyielding dedication to their occupation, which often results in a never-ending cycle of work without any perceived progression in their lives. Another tweet that captures this sentiment reads, "Drowning in more work and poverty despite trying harder and harder," pointing to a sense of despair and dissatisfaction that comes with work that is both disproportionately demanding and inadequately rewarding. Critics also note how the pursuit of hard-earned money could potentially create physical and mental health risks due to the unrelenting pressure and stress that these jobs might impose. Conversely, those in favor of earning money with less difficulty contend that it's crucial to harness one's strengths and passions to create opportunities that yield financial success without the need for excessive labor. The debate revolves around the concept that people should seek out ways to work smarter, not harder, especially if it means a healthier and more fulfilling lifestyle.
----------------------------------------------------------------------------------------
Split 3, tokens 137, triggered by: 0.85
In fact, the notion of a "vicious cycle," often attributed to those chasing hard-earned money, is juxtaposed with an idealized image of someone operating in their zone of excellence. Confidently focused on their strengths, such individuals are depicted as enjoying a more relaxed and less stressful work environment, one in which they can thrive without the need for never-ending overtime or self-sacrifice. Some tweets even extend this sentiment to the broader socio-economic context, observing how wealth is not merely derived from manual labor or high-paying positions requiring extraordinary work hours. The tweets emphasize the importance of cultivating an entrepreneurial spirit and a penchant for innovative thinking, especially in the modern digital age.
----------------------------------------------------------------------------------------
Split 4, tokens 249, triggered by: 0.86
One user writes, "Too hard-earned money isn't worth it. Learn how to make money using your brain, not your body," while another suggests, "Love will flow towards those who are not lacking in love, and money will flow towards those who are not lacking in money!" While some of the discussions take a somewhat passive-aggressive view, others acknowledge that financial security and comfort might not always be possible for everyone. In a more realistic tone, a user remarks, "If life were so easy that diligence led to wealth, then the world's richest person would be the best worker bee. But that's not the case." This acknowledgment underscores the complexities of the economy and the role that factors like luck, connections, and a rapidly evolving job market can play in financial success. Some users are quick to criticize the notion that earning money the hard way should be avoided, with one tweet expressing, "The person who advises you to avoid hard-earned money is likely a scammer who profits off providing emotional value in exchange for exploitation." Others argue that while it's essential to find enjoyment and fulfillment in one's work, it's crucial not to shun or belittle those who choose to work in physically demanding or high-paying industries.
----------------------------------------------------------------------------------------
Split 5, tokens 117, triggered by: 0.87
Overall, the Weibo discussions offer a fascinating insight into the complexities of the modern Chinese labor market and the work-life balance that people strive to achieve. As in many countries, striking the right balance between work and play is an ongoing challenge for many Chinese citizens. However, the conversations on Weibo signal an increasing awareness of the importance of finding meaningful, fulfilling, and financially rewarding work that doesn't necessitate excessive sacrifice or sufferance. In the end, as one user succinctly puts it, "Make sure you're earning your money in a way that brings you joy and satisfaction.
----------------------------------------------------------------------------------------
Split 6, tokens 171, triggered by: 0.82
That's the only way to ensure that your life doesn't become a never-ending cycle of hard work without any tangible progress." In this context, social media discussions focusing on the trials and tribulations of earning money serve not only as an outlet for venting frustrations but also as a means of promoting dialogue and shared understanding about the challenges faced by workers across all industries. These virtual conversations sparked by tweets and in-depth discussions likely resonate with a wide swath of Chinese citizens struggling to navigate the complexities of balancing a career that pays well with one that brings them joy, fulfillment, and a sense of purpose. As the discussions on Weibo continue to evolve and unfold, it is evident that the discourse around work, money, and life satisfaction holds the potential to inspire meaningful change and shift societal attitudes towards a more holistic, balanced, and humane understanding of success and prosperity.
----------------------------------------------------------------------------------------
Split 7, tokens 72, triggered by: 0.80
--- Note: The translated tweets and user quotes from Chinese to English were used as the foundation for the long-form news article. The author tried to maintain the integrity of the original content in the translation while adapting it to fit a journalistic format. No inaccuracies were introduced during translation, and the opinion-based nature of the original content was preserved while maintaining objectivity.
----------------------------------------------------------------------------------------
Split 8, tokens 23, triggered by: final split
Heart count: 0/2 Note: The author did not include any Chinese characters in the final response. Collapse
----------------------------------------------------------------------------------------
%% Cell type:code id: tags:
``` python
```
docs/examples/text_semantic_split.ipynb deleted 100644 → 0
+ 0
− 179
View file @ afc97d93
This diff is collapsed.
poetry.lock
+ 10
− 408
View file @ 57f024c9
This diff is collapsed.
pyproject.toml
+ 2
− 1
View file @ 57f024c9
......@@ -35,6 +35,7 @@ regex = "^2023.12.25"
torchvision = { version = "^0.16.2", optional = true}
pillow = { version= "^10.2.0", optional = true}
tiktoken = "^0.6.0"
matplotlib = { version="^3.8.3", optional = true}
[tool.poetry.extras]
hybrid = ["pinecone-text"]
......@@ -42,6 +43,7 @@ fastembed = ["fastembed"]
local = ["torch", "transformers", "llama-cpp-python"]
pinecone = ["pinecone-client"]
vision = ["torch", "torchvision", "transformers", "pillow"]
processing = ["matplotlib"]
[tool.poetry.group.dev.dependencies]
ipykernel = "^6.25.0"
......@@ -53,7 +55,6 @@ pytest-xdist = "^3.5.0"
mypy = "^1.7.1"
types-pyyaml = "^6.0.12.12"
types-requests = "^2.31.0"
matplot = "^0.1.9"
[build-system]
requires = ["poetry-core"]
......
semantic_router/splitters/rolling_window.py
+ 4
− 1
View file @ 57f024c9
......@@ -215,7 +215,10 @@ class RollingWindowSplitter(BaseSplitter):
try:
from matplotlib import pyplot as plt
except ImportError:
logger.warning("Plotting is disabled. Please `pip install matplotlib`.")
logger.warning(
"Plotting is disabled. Please `pip install "
"semantic-router[processing]`."
)
return
if not self.plot_splits:
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

:feet::heart: Strive to be the person your dogs believe you are :heart::feet: