Summary:
This diff enables building the eden watcher by linking in thrift and
its various dependencies.
To support building in-fbsource and in the github repo, a `maybe_shipit_dir`
function is used to setup a symlink to the `eden` and `fboss/common` dirs (this
mirrors the shipit configuration for this project: we cannot simply run shipit
because we have to build on mac and windows and shipit requires Hack, and that
does not support those platforms).
I tried to persuade cmake to let me build this without the use of a symlink but
found it too difficult to teach everything about the path mapping. The
symlinks aren't terrible, but are the reason why this diff also updates some
`.gitinore` files that are seemingly unrelated to this diff.
This diff changes a couple of build/link options: without them the end product
fails to link either due to implicit/unilateral enablement of UBSAN in some of
the deps, or because warning->error promotion is turned on.
This diff includes a copy of the `ThriftCppLibrary.cmake` file from the fboss
repo. This should get centralized and shipit'ed out into the places that
consume it. That can be done when someone gets around to doing the same for
the `FindGlog.cmake` file and doesn't need to hold up this diff.
Reviewed By: simpkins
Differential Revision: D13486486
fbshipit-source-id: 3bb5b011771b2a87618147ca019b4e50a8e0aaf2
Summary:
This patch enables BST stats by default, and works as follows:
- it uses quantile stats to maintain per queue per port stats.
- these stats are exposed to ODS.
- experiments showed that a significant CPU cycles were spent in doing
bcm_cosq_bst_stat_sync which syncs the hw stat value to sw copy.
For example, calling bcm_cosq_bst_stat_sync while retrieving BST stats
for for every port/every queue/every second resulted into CPU utilization
going up from ~ 50% for Wedge 100 to ~100-120%. Thus, this patch calls
bcm_cosq_bst_stat_sync only once per minute and then retrieves all BST stats.
- The above optimization also helps when bufferstatlogging is enabled.
- when bufferstatlogging is enabled, the BST stats will be retrieved every
second for scuba logging, and quantile stats will also be exported every
second.
- The bufferstatlogging is enabled using fboss CLI (over thrift).
We no longer need enable_fine_grained_buffer_stats option. But, a new option
viz. update_bststats_interval_s is introduced that keeps the frequency of BST
stats updates configurable.
The virtual memory overhead of always maintaining BST stats is < 5%, and
physical memory overhead is < 0.1%. Thanks to the sync optimization above,
the CPU overhead is negligible. Refer Test Plan for raw data and math.
An alternative is to explore using TimeSeriesMinMax (to maintain max over a
time window) and TLCounter (to expose stats to ODS) instead of quantile stats.
One benefit of using quantile stats is that we don't need to maintain our own
TimeSeriesMinMax but can rely on common facebook wide infra. Moreoever,
quantile stats interface is (arguably) simpler than using combination of
TimeSeriesMinMax + TlCounter'.
A separate diff against configerator is sent out to expose the new bst_stats: D8902636
Differential Revision: D8764329
fbshipit-source-id: 78db3de5340497f2f5780863035dd9ff28dcd79b
Summary:
We call this function in Eden, so make it available in the fb303 stubs that
we include in the github repository.
Differential Revision: D12813738
fbshipit-source-id: 1432e751ce676410e9d2e589274c7c1dc2c45321
Summary:
Solves an issue building eden GitHub version:
eden/fs/service/EdenServer.cpp:1044:34: error: ‘class facebook::stats::ServiceData’ has no member named ‘addStatValue’
stats::ServiceData::get()->addStatValue(
^~~~~~~~~~~~
Reviewed By: bolinfest, simpkins
Differential Revision: D9029274
fbshipit-source-id: c0e5ea492c1c38f16bfae4b08d41538a2443450f
Summary:
This removes the TARGETS files from the eden github repository. The
open source buck build has been failing for several months, since buck
removed support for the thrift_library() rule.
I will potentially take a stab at adding CMake build support for Eden
at some point in the future.
Reviewed By: chadaustin
Differential Revision: D6893233
fbshipit-source-id: e6023094a807cf481ac49998c6f21b213be6c288
Summary:
aliveSince() has a misleading name and I misunderstood.
The correct implementation just returns the timestamp of when
the process started as opposed to my understanding (the number
of seconds since the process started).
Reviewed By: linhbui
Differential Revision: D6789987
fbshipit-source-id: db60bf00f6b07e31d91dfc8ed06d1c1cbe172a64
Summary:
fb303 is not open source, but fboss depends on it. So, the
open source version of fboss uses its own private, mostly
stubbed out version of fb303 (in fboss/common/...).
With the changes to QsfpCache.cpp, we now depend on aliveSince()
fb303 primitive and the open source started spewing errors
about it not being implemented.
This patch implements aliveSince() for the skeleton open source
fb303 but we'll have to figure out a better, longer term fix
(e.g., open source all of fb303).
Reviewed By: aeckert
Differential Revision: D6731321
fbshipit-source-id: 6acdcd68053e7bfcada2cfd62aaa3aecf3dec3e0
Summary:
My recent stack of diffs introduced a handful of oss bugs :(.
1. VLOG_EVERY_MS is an internal function.
fix: added an oss stub version
2. Forgot to add QsfpCache.cpp to CMakeLists.txt
fix: add it
3. vanilla gcc5.4 has a bug where it can't handle lambdas like:
[this]() {
memberFn();
}
and instead you need to write like:
[this]() {
this->memberFn();
}
fix: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67274 says it should
be fixed in gcc7, but I added 'this->' in the meantime.
4. The base fb303.thrift file we have in open source does not declare
aliveSince, which we now call on qsfp_service.
fix: add it to fb303.thrift
Reviewed By: ninas
Differential Revision: D6627705
fbshipit-source-id: 2100783df1ea0e9af0fed66e4e24ef85b71fc7e9
Summary:
This is a codemod to change from using @/ to // in basic cases.
- TARGETS files with lines starting with @/ (but excluding @/third-party:
- autodeps lines in source and TARGETS files ( (dep|manual)=@/ ), excluding @/third-party
- Targets in string macros
The only thing left of the old format should be @/third-party:foo:bar
drop-conflicts
Reviewed By: ttsugriy
Differential Revision: D6605560
fbshipit-source-id: 17d3a196b91045f0db5ee2a5afad467b6344be0b
Summary:
In this diff, the main goal is to change the old `link_state.down` to `link_state.flap`. However, the tricky part is how can we use the normalized
portName(ethX/Y/Z) instead of portX as the key name. Since, the initialization of wedge_agent will go through two stage genrally:
In such case, we need to support changing the portName for PortStats after it's created.
My last diff(D5695076) is trying to delete all the TLTimeseries of all the threads in PortUpdateHandler observer, while ThreadLocalStats will also
try to unregister(delete) the TLTimeseries before we shut down wedge_agent. Therefore, we saw this:
```
Sep 16 19:09:52 rsw1fs.17.prn2.facebook.com wedge_agent[1884398]: F0916 19:09:52.716562 1884650 ThreadLocalStats-defs.h:495] Check failed: numErased == 1 attempted to unregister a stat that was not registered
```
So I guess probably it's not a good idea to delete a TLTimeseries of some thread in a different thread, unless you can get the global lock like ThreadLocalStats
does. Thus, in this new diff, I kinda walk around it:
Differential Revision: D5914457
fbshipit-source-id: f7d47d2ececfabbab8fdc222a0f82db4184230f9
Summary:
Depend on D4492997
Currently, we assume that changes applied to HwSwitch always succeed (or the process exits). There is a case where we want to keep moving despite failures: when we get excessive number of route (probably) because of some misconfiguration. This diff is a step towards taking care of such situation.
We change the interface between SwSwitch and HwSwitch so that HwSwitch, when asked to apply delta of old and new state, returns either the new state if it succeeds, or returns a "pruned version of new state" if some elements of new state it was not able to apply.
SwSwitch now keeps track of two states, a state that has been applied in the hardware, and another one that is desired in hardware. SwSwitch strives to catch applied state to the desired state (while desired state is also changes as move state updates are arriving).
One issue with this diff: Warmboot is not preserving "unapplied" routes, i.e., unapplied routes get lost across warmboots. This will be fixed in a seperate diff.
One naming issue: I thought about keeping SwSwitch::getState() name intact, but thought that it is not a good idea to keep distinction of states opaque to callers. So, decided to change the name.
Reviewed By: ninas
Differential Revision: D4405751
fbshipit-source-id: c933b4418445ae49d5b8e21dea95e437a8e1593d
Summary:
Setting default value to empty causes trouble on server if client is old as
server will think that value is set to empty and it tries to parse empty value.
This field is not being used anywhere except in `openr` for now and openr is
fully compatible with this change. Not setting default value will give us as
`__isset.ifName` to `false` instead of being empty string, which is desired
behaviour.
(Note: this ignores all push blocking failures!)
Reviewed By: anantdeepak
Differential Revision: D4934119
fbshipit-source-id: 4dd2e9307c659f1b6ccaf583964efa8df6041d24
Summary:
Logical ids are so 2014. We have been thinking
about emitting stats using normalized names for some time.
Approach wise, I picked the following
- If port does not have a name - default to using port ID
- In agent if a port name get set or updated. Re init BCMPort stats. Basically
this means that the moment config gets applied, we would start emitting
normalized interface stats.
Two additional pieces of work here are
i) Update interface based alerts to also look at these new normalized
interface name stats - t17052196
ii) Update cubism to look at these new names - t17052255
I won't land till at least i) is fixed
Differential Revision: D4815561
fbshipit-source-id: c088b5a8981ea40429fedc8570652c376a374be3
Summary:
We are using `BinaryAddress` heavily within fboss codebase to represent a
nexthop as well as an IPAddres in wire format. Nexthops will need to have
a scoping identifier (iface name) for link-local address inorder to resolve
it properly.
We do this in OpenR and well tested on Arista. This seems bit hacky but it
saves a lot of time. In future we would like to have separate network util for
fboss and openr and we can get rid of this hack at that time.
Differential Revision: D4710768
fbshipit-source-id: 72092935f3738b3fb88a422462d1c995cc0acf18
Summary:
This adds a common/ directory with stub files required for building
eden. These stubs are the same ones as used in the fboss repository:
https://github.com/facebook/fboss/
Our goal is to eventually create an opensource fb303 repository with
complete implementations of these files, but for now these basic stubs
allow building eden. (Most of the underlying stats code that supports
fb303 is already available in folly/stats/ in the folly repository.)
Test Plan:
Tested building eden on Ubuntu 16.04
