You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 22:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue Apr 14, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77892994439,3cyclictest191727-21kworker/u8:3+flush-259:009:10:002
77887994440,2cyclictest160781-21fwupd08:30:020
77887994329,12cyclictest176671-21diskstats08:45:140
77887994329,12cyclictest176671-21diskstats08:45:130
77892994234,6cyclictest124216-21latency_hist07:55:012
77892994135,4cyclictest252-21systemd-journal09:00:002
77897993530,3cyclictest237229-21kworker/u8:1+flush-259:012:05:013
77889993531,2cyclictest310898-21kworker/u8:3+events_unbound11:10:001
77887993520,3cyclictest252-21systemd-journal11:39:080
77897993429,3cyclictest222776-21kworker/u8:0+flush-259:009:50:013
77887993428,4cyclictest419069-21ntpq12:35:220
77887993327,4cyclictest331497-21latency_hist11:15:010
77889993126,3cyclictest252-21systemd-journal09:46:081
77889993124,5cyclictest252-21systemd-journal08:30:001
77887993124,5cyclictest226998-21ntpq09:30:200
7788799311,29cyclictest1-21systemd11:59:080
7788799310,30cyclictest1-21systemd08:37:070
77892993026,3cyclictest39746-21kworker/u8:3+flush-259:007:25:172
77892993025,3cyclictest310898-21kworker/u8:3+flush-259:011:05:132
77892993025,3cyclictest237229-21kworker/u8:1+events_unbound10:40:012
77889993024,4cyclictest185613-21ntpq08:50:211
77889992821,5cyclictest419983-21latency_hist12:40:011
77892992722,3cyclictest39747-21kworker/u8:4+flush-259:007:55:222
77889992723,3cyclictest259188-21apt-get10:05:121
77889992721,4cyclictest155609-21latency_hist08:25:011
77887992720,5cyclictest134541-21latency_hist08:05:000
77892992622,3cyclictest331516-21kworker/u8:0+events_unbound11:35:022
77892992620,4cyclictest252-21systemd-journal08:14:082
77892992619,5cyclictest346587-21ntpq11:25:242
77889992623,2cyclictest222776-21kworker/u8:0+flush-259:009:55:251
77897992519,4cyclictest280003-21latency_hist10:25:003
77897992519,4cyclictest150476-21latency_hist08:20:003
77897992519,4cyclictest118256-21ntpq07:45:223
77892992521,3cyclictest237229-21kworker/u8:1+flush-259:010:05:142
77892992520,4cyclictest171058-21kworker/u8:1+flush-259:008:45:002
77889992521,3cyclictest237229-21kworker/u8:1+flush-259:009:50:261
77889992519,4cyclictest252-21systemd-journal12:18:071
77897992420,3cyclictest331516-21kworker/u8:0+flush-259:011:30:213
77897992420,3cyclictest237229-21kworker/u8:1+flush-259:010:50:163
77897992420,3cyclictest129623-21apt-get08:00:123
77892992420,3cyclictest237229-21kworker/u8:1+flush-259:010:50:192
77892992419,4cyclictest222776-21kworker/u8:0+flush-259:009:44:232
77892992419,3cyclictest248656-21kworker/u8:2+flush-259:010:10:222
77892992419,3cyclictest237229-21kworker/u8:1+flush-259:011:40:252
77889992418,4cyclictest509-21rs:main1
77887992420,3cyclictest310898-21kworker/u8:3+flush-259:011:20:140
77887992418,4cyclictest252-21systemd-journal07:25:000
77897992319,3cyclictest252-21systemd-journal07:39:013
77897992317,4cyclictest248604-21awk09:55:003
77892992319,3cyclictest237229-21kworker/u8:1+events_unbound12:10:092
77892992319,3cyclictest237229-21kworker/u8:1+events_unbound11:00:142
77892992319,3cyclictest171058-21kworker/u8:1+events_unbound08:50:132
77892992318,3cyclictest39748-21kworker/u8:5+events_unbound07:10:202
77892992317,4cyclictest383964-21kworker/u8:0+events_unbound12:19:102
77889992318,3cyclictest207442-21kworker/u8:4+flush-259:009:15:171
77892992219,2cyclictest379464-21irqcore12:00:162
77892992218,3cyclictest39747-21kworker/u8:4+events_unbound07:49:062
77892992217,3cyclictest264774-21kworker/u8:3+events_unbound10:34:352
77892992216,1cyclictest108782-21munin-run07:40:002
77892992215,5cyclictest87273-21ntpq07:15:192
77889992218,3cyclictest82923-21kworker/u8:0+flush-259:007:45:241
77889992218,3cyclictest383964-21kworker/u8:0+flush-259:012:30:251
77889992215,5cyclictest341022-21ntpq11:20:231
77887992218,3cyclictest191727-21kworker/u8:3+flush-259:009:30:000
77897992117,3cyclictest252-21systemd-journal10:19:593
77892992117,3cyclictest373820-21kworker/u8:4+flush-259:012:05:262
77892992117,3cyclictest300621-21kworker/u8:2+events_unbound11:10:162
77892992117,2cyclictest0-21swapper/210:19:592
77892992116,4cyclictest274872-21kworker/u8:0+events_unbound10:25:282
77892992116,3cyclictest39747-21kworker/u8:4+flush-259:008:25:202
77892992116,3cyclictest39747-21kworker/u8:4+flush-259:008:18:072
77892992115,4cyclictest341866-21latency_hist11:25:012
77889992118,2cyclictest0-21swapper/107:40:001
77889992114,5cyclictest372712-21ntpq11:50:211
77887992118,2cyclictest352929-21idleruntime-cro11:34:590
77887992117,3cyclictest352985-21kworker/u8:2+flush-259:011:47:080
77887992117,3cyclictest186796-21apt-get08:55:140
77897992015,3cyclictest509-21rs:main3
77892992017,2cyclictest191727-21kworker/u8:3+flush-259:009:15:012
77892992016,3cyclictest404165-21kworker/u8:3+flush-259:012:30:152
77892992016,3cyclictest264774-21kworker/u8:3+events_unbound10:20:192
77892992016,3cyclictest237229-21kworker/u8:1+events_unbound09:50:142
77892992014,4cyclictest88144-21kworker/u8:1+flush-259:007:20:052
77889992017,2cyclictest509-21rs:main1
77889992016,3cyclictest274872-21kworker/u8:0+events_unbound10:40:221
77889992016,3cyclictest207442-21kworker/u8:4+flush-259:009:25:001
77897991916,2cyclictest274872-21kworker/u8:0+flush-259:010:34:593
77897991914,4cyclictest88125-21latency_hist07:20:023
77897991912,5cyclictest134537-21apt-get08:05:003
77892991915,3cyclictest39747-21kworker/u8:4+events_unbound08:02:072
77892991915,3cyclictest383964-21kworker/u8:0+events_unbound12:30:012
77892991915,3cyclictest181397-21kworker/u8:2+events_unbound09:30:222
77892991915,2cyclictest0-21swapper/209:30:012
77892991913,4cyclictest252-21systemd-journal10:05:002
77892991912,5cyclictest252-21systemd-journal08:40:002
77889991916,2cyclictest336673-21idleruntime-cro11:20:001
77889991915,3cyclictest237229-21kworker/u8:1+flush-259:010:30:241
77889991915,3cyclictest160774-21kworker/u8:0+flush-259:008:35:201
77889991914,4cyclictest285172-21latency_hist10:30:001
77889991912,5cyclictest82978-21awk07:15:011
77887991915,3cyclictest39747-21kworker/u8:4+flush-259:008:15:180
77887991914,3cyclictest509-21rs:main0
77887991914,3cyclictest252-21systemd-journal09:25:010
77897991815,2cyclictest341894-21gpgv11:25:003
77897991815,2cyclictest268899-21nvmesmart_nvme010:10:243
77897991814,3cyclictest326625-21apt-get11:10:123
77897991813,4cyclictest368424-21latency_hist11:50:013
77897991812,4cyclictest258923-21latency_hist10:05:013
77897991810,7cyclictest237229-21kworker/u8:1+flush-259:011:04:593
77892991815,2cyclictest363289-21kworker/u8:3+flush-259:011:55:002
77892991815,2cyclictest253775-21sh10:00:012
77892991815,2cyclictest207903-21irqcore09:15:172
77892991814,3cyclictest39747-21kworker/u8:4+flush-259:007:44:232
77892991814,3cyclictest39746-21kworker/u8:3+flush-259:007:30:212
77892991814,3cyclictest237229-21kworker/u8:1+events_unbound10:45:222
77892991814,3cyclictest181397-21kworker/u8:2+flush-259:009:39:132
77892991813,4cyclictest274872-21kworker/u8:0+events_unbound10:42:102
77892991813,4cyclictest160774-21kworker/u8:0+flush-259:008:50:012
77892991813,4cyclictest160774-21kworker/u8:0+flush-259:008:50:002
77892991811,1cyclictest0-21swapper/209:50:002
77889991815,2cyclictest347412-21idleruntime-cro11:30:001
77889991813,3cyclictest252-21systemd-journal11:42:081
77889991812,4cyclictest311141-21apt-get10:55:111
77887991814,2cyclictest316702-21irqcore11:00:170
77887991814,2cyclictest1-21systemd09:14:090
77887991813,4cyclictest123338-21ntpq07:50:220
77897991714,1cyclictest0-21swapper/311:40:003
77892991713,3cyclictest414380-21kworker/u8:2+events_unbound12:36:102
77892991713,2cyclictest0-21swapper/208:09:592
77892991712,4cyclictest252-21systemd-journal08:34:542
77889991714,2cyclictest39747-21kworker/u8:4+events_unbound07:50:171
77889991714,2cyclictest228576-21irqcore09:35:191
77889991713,3cyclictest82923-21kworker/u8:0+events_unbound07:15:181
77889991713,3cyclictest373820-21kworker/u8:4+flush-259:011:55:471
77889991713,3cyclictest348096-21apt-get11:30:111
77887991715,2cyclictest39747-21kworker/u8:4+events_unbound08:05:170
77887991715,1cyclictest389050-21cut12:10:010
77887991714,2cyclictest274872-21kworker/u8:0+flush-259:010:40:170
77887991713,3cyclictest273958-21ntp_kernel_pll_10:15:220
77887991712,4cyclictest252-21systemd-journal08:44:110
77887991712,3cyclictest243441-21latency_hist09:50:000
77897991613,2cyclictest186527-21sh08:55:003
77897991612,3cyclictest129401-21kworker/u8:1+flush-259:008:15:003
77892991613,2cyclictest394221-21kworker/u8:2+flush-259:012:25:012
77892991612,3cyclictest237229-21kworker/u8:1+events_unbound10:55:232
77889991614,1cyclictest39747-21kworker/u8:4+flush-259:007:35:001
77889991613,2cyclictest141388-21kthreadcore08:10:181
77889991611,4cyclictest254009-21/usr/sbin/munin10:00:241
77887991614,1cyclictest60350irq/132-enp0s29f1:rx-011:40:260
77887991613,2cyclictest61150irq/133-enp0s29f1:tx-011:50:240
77887991613,2cyclictest60350irq/132-enp0s29f1:rx-011:25:190
77887991613,2cyclictest60350irq/132-enp0s29f1:rx-010:55:130
77887991613,2cyclictest60350irq/132-enp0s29f1:rx-010:25:120
77887991613,2cyclictest60350irq/132-enp0s29f1:rx-010:21:290
77887991613,2cyclictest207891-21irqcore09:15:170
77887991612,3cyclictest103856-21/usr/sbin/munin07:35:210
77897991512,2cyclictest222776-21kworker/u8:0+flush-259:010:00:013
77897991512,2cyclictest191727-21kworker/u8:3+flush-259:009:25:173
77897991511,3cyclictest88396-21apt-get07:20:123
77897991511,3cyclictest171310-21/usr/sbin/munin08:40:233
7789299159,4cyclictest237229-21kworker/u8:1+flush-259:011:58:492
7788999158,5cyclictest77821-21latency_hist07:10:011
77889991512,2cyclictest303155-21kthreadcore10:45:181
77889991510,3cyclictest39747-21kworker/u8:4+flush-259:007:40:171
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-012:25:240
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-010:10:240
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-010:05:230
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:180
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-009:50:200
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-009:35:180
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-009:05:200
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-008:50:160
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-008:25:210
77887991513,1cyclictest60350irq/132-enp0s29f1:rx-008:21:070
77887991513,1cyclictest412533-21kthreadcore12:30:200
77887991510,4cyclictest181397-21kworker/u8:2+flush-259:009:04:550
7789799148,4cyclictest102705-21ntpq07:30:213
77897991411,2cyclictest291609-21kthreadcore10:35:183
77897991411,1cyclictest0-21swapper/311:30:003
77889991411,2cyclictest98462-21sh07:30:011
77889991411,2cyclictest331516-21kworker/u8:0+flush-259:011:35:181
7788799147,5cyclictest513-21gdbus08:14:590
7788799147,5cyclictest252-21systemd-journal11:09:070
77887991413,1cyclictest60350irq/132-enp0s29f1:rx-009:40:190
77887991412,2cyclictest60350irq/132-enp0s29f1:rx-007:19:110
77887991412,1cyclictest60350irq/132-enp0s29f1:rx-012:15:240
77887991412,1cyclictest60350irq/132-enp0s29f1:rx-012:10:160
77887991412,1cyclictest60350irq/132-enp0s29f1:rx-012:00:140
77887991412,1cyclictest60350irq/132-enp0s29f1:rx-011:15:250
77887991412,1cyclictest60350irq/132-enp0s29f1:rx-007:55:160
77887991410,3cyclictest39746-21kworker/u8:3+flush-259:007:32:350
77887991410,3cyclictest116478-21kthreadcore07:45:170
7789799139,3cyclictest222776-21kworker/u8:0+flush-259:009:42:393
7789799139,3cyclictest207442-21kworker/u8:4+flush-259:009:18:393
7789799139,3cyclictest181397-21kworker/u8:2+flush-259:009:10:123
77897991311,1cyclictest374422-21irqcore11:55:163
7788999139,3cyclictest201119-21ntp_kernel_err09:05:211
7788999138,3cyclictest0-21swapper/108:40:241
7788999137,4cyclictest269694-21latency_hist10:15:011
77889991310,2cyclictest0-21swapper/111:00:171
77887991311,1cyclictest60350irq/132-enp0s29f1:rx-010:51:070
77887991311,1cyclictest60350irq/132-enp0s29f1:rx-007:25:160
77887991311,1cyclictest0-21swapper/007:10:160
7789799127,3cyclictest404165-21kworker/u8:3+flush-259:012:30:183
77897991210,1cyclictest0-21swapper/312:15:203
7789299129,2cyclictest352104-21ntp_offset11:30:212
7789299128,3cyclictest331516-21kworker/u8:0+flush-259:011:15:212
7788999129,2cyclictest237229-21kworker/u8:1+flush-259:010:35:141
7788999129,2cyclictest135638-21irqcore08:05:171
7788999128,3cyclictest181397-21kworker/u8:2+flush-259:009:00:361
7788999127,4cyclictest509-21rs:main1
7788999126,4cyclictest274064-21wc10:15:231
77889991210,1cyclictest0-21swapper/112:05:181
77889991210,1cyclictest0-21swapper/111:15:001
77887991211,1cyclictest60350irq/132-enp0s29f1:rx-009:55:240
77887991210,1cyclictest60350irq/132-enp0s29f1:rx-010:35:150
7789799119,1cyclictest0-21swapper/308:30:213
7789799118,2cyclictest150808-21cpuspeed_turbos08:20:133
7789799117,3cyclictest207442-21kworker/u8:4+flush-259:009:35:313
7789799117,2cyclictest237229-21kworker/u8:1+flush-259:012:05:253
7789299110,9cyclictest77821-21latency_hist07:10:002
7788999119,1cyclictest0-21swapper/108:05:001
7788999118,2cyclictest383085-21ntp_offset12:00:211
7788799119,1cyclictest60350irq/132-enp0s29f1:rx-007:40:170
7788999108,1cyclictest390319-21kthreadcore12:10:161
7788999108,1cyclictest191701-21idleruntime-cro09:00:011
7788999108,1cyclictest0-21swapper/109:10:131
7788999107,2cyclictest237229-21kworker/u8:1+flush-259:012:20:281
7788999106,3cyclictest237229-21kworker/u8:1+flush-259:011:45:171
7788999106,2cyclictest0-21swapper/109:25:211
778899997,1cyclictest0-21swapper/109:40:191
778899984,3cyclictest147952-21cut08:15:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional