You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue May 26, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
442761994134,5cyclictest707454-21latency_hist11:25:000
442761994037,1cyclictest1-21systemd11:52:520
442770993831,5cyclictest447885-21latency_hist07:15:002
442772993731,4cyclictest493392-21ntpq07:55:213
442770993528,5cyclictest697153-21latency_hist11:15:002
442766993530,4cyclictest717801-21latency_hist11:35:001
442766993430,3cyclictest770998-21kworker/u8:4+flush-259:012:25:151
442761993327,4cyclictest716942-21ntpq11:30:210
442772993226,4cyclictest665874-21/usr/sbin/munin10:45:143
442770993213,14cyclictest680222-21ntpq10:55:212
442766993228,3cyclictest650357-21kworker/u8:3+flush-259:010:30:211
442772993024,4cyclictest618774-21latency_hist10:00:013
442770993024,5cyclictest0-21swapper/210:30:002
442761993026,3cyclictest603480-21kworker/u8:1+flush-259:010:05:000
442761993026,3cyclictest427569-21kworker/u8:3+flush-259:007:10:150
442761993024,4cyclictest520620-21cp08:25:010
442766992924,4cyclictest442682-21kworker/u8:0+flush-259:007:45:011
442766992825,2cyclictest712347-21kworker/u8:1+flush-259:011:55:151
442761992822,4cyclictest545681-21ntpq08:45:200
44276199281,26cyclictest1-21systemd11:46:290
442772992724,2cyclictest755225-21kworker/u8:3+flush-259:012:10:163
442772992721,4cyclictest649854-21cat10:30:013
442766992723,3cyclictest536951-21kworker/u8:0+events_unbound09:15:221
442766992722,4cyclictest488210-21ntpq07:50:211
442766992721,4cyclictest665635-21latency_hist10:45:011
442766992624,2cyclictest650357-21kworker/u8:3+flush-259:011:10:221
442766992623,2cyclictest499283-21kworker/u8:1+flush-259:008:25:211
442766992622,3cyclictest781827-21apt-get12:35:111
442766992622,3cyclictest691155-21apt-get11:06:361
442766992622,3cyclictest613637-21kworker/u8:0+events_unbound10:15:181
442766992622,3cyclictest561629-21kworker/u8:2+events_unbound09:35:121
442761992620,4cyclictest535371-21ntpq08:35:210
442772992522,2cyclictest755225-21kworker/u8:3+flush-259:012:15:263
442772992521,3cyclictest499283-21kworker/u8:1+events_unbound08:55:143
442772992520,4cyclictest509-21rs:main3
442766992522,2cyclictest650357-21kworker/u8:3+events_unbound10:55:191
442766992521,3cyclictest561629-21kworker/u8:2+events_unbound09:20:221
442766992520,3cyclictest335557-21systemd-journal11:15:281
442761992520,3cyclictest211rcuc/008:00:000
442772992420,3cyclictest738345-21kworker/u8:0+flush-259:012:10:013
442772992420,3cyclictest738345-21kworker/u8:0+flush-259:012:10:003
442766992421,2cyclictest442682-21kworker/u8:0+events_unbound07:24:291
442766992421,2cyclictest442682-21kworker/u8:0+events_unbound07:24:281
442766992420,3cyclictest567034-21kworker/u8:3+flush-259:009:10:001
442766992419,4cyclictest655740-21diskstats10:35:151
442761992418,4cyclictest499633-21/usr/sbin/munin08:05:180
442772992320,2cyclictest738345-21kworker/u8:0+events_unbound12:00:183
442772992320,2cyclictest650357-21kworker/u8:3+events_unbound11:35:143
442772992319,3cyclictest650357-21kworker/u8:3+events_unbound10:35:183
442772992319,3cyclictest582704-21kworker/u8:3+flush-259:009:50:003
442772992319,3cyclictest582704-21kworker/u8:3+flush-259:009:50:003
442772992316,5cyclictest686261-21latency_hist11:05:013
442770992318,3cyclictest776662-21apt-get12:30:112
442766992320,2cyclictest702539-21kworker/u8:0+flush-259:011:25:201
442766992320,2cyclictest442682-21kworker/u8:0+events_unbound07:48:281
442766992319,3cyclictest660538-21kworker/u8:0+events_unbound11:03:291
442766992319,3cyclictest561629-21kworker/u8:2+events_unbound09:40:221
442766992319,3cyclictest552010-21apt-get08:55:091
442766992319,3cyclictest536951-21kworker/u8:0+events_unbound09:30:151
442766992319,3cyclictest510809-21kworker/u8:2+flush-259:008:30:111
442766992319,3cyclictest442682-21kworker/u8:0+flush-259:008:10:211
442766992319,3cyclictest442682-21kworker/u8:0+flush-259:007:19:291
442766992318,4cyclictest644564-21latency_hist10:25:011
442772992219,2cyclictest536951-21kworker/u8:0+flush-259:009:15:173
442772992218,3cyclictest692075-21base6411:10:013
442770992212,8cyclictest722937-21latency_hist11:40:002
442766992219,2cyclictest738345-21kworker/u8:0+events_unbound12:00:241
442766992219,2cyclictest650357-21kworker/u8:3+events_unbound11:45:231
442766992219,2cyclictest531183-21kworker/u8:3+flush-259:009:00:251
442766992219,2cyclictest499283-21kworker/u8:1+events_unbound08:15:241
442766992218,3cyclictest755225-21kworker/u8:3+events_unbound12:15:151
442766992218,3cyclictest582704-21kworker/u8:3+events_unbound10:01:301
442766992218,3cyclictest510809-21kworker/u8:2+events_unbound08:35:151
442766992217,3cyclictest629306-21kworker/u8:4+flush-259:010:25:231
442766992216,4cyclictest614036-21diskstats09:55:131
442766992216,4cyclictest608640-21/usr/sbin/munin09:50:131
442766992215,5cyclictest755281-21latency_hist12:10:001
442761992215,5cyclictest335557-21systemd-journal08:25:000
44276199220,20cyclictest1-21systemd08:16:280
442772992118,2cyclictest603480-21kworker/u8:1+flush-259:010:05:253
442772992118,2cyclictest458212-21kworker/u8:3+events_unbound07:30:183
442772992117,3cyclictest650357-21kworker/u8:3+events_unbound11:05:243
442772992117,3cyclictest644511-21kworker/u8:1+events_unbound10:43:303
442772992117,3cyclictest629306-21kworker/u8:4+events_unbound10:18:303
442772992117,2cyclictest442682-21kworker/u8:0+events_unbound08:15:143
442770992117,2cyclictest515271-21latency_hist08:20:002
442770992114,5cyclictest750125-21latency_hist12:05:002
442766992118,2cyclictest499283-21kworker/u8:1+events_unbound08:52:291
442766992118,2cyclictest448555-21kworker/u8:1+events_unbound07:30:111
442766992117,3cyclictest603480-21kworker/u8:1+events_unbound09:45:121
442766992117,3cyclictest603480-21kworker/u8:1+events_unbound09:45:111
442766992114,4cyclictest509-21rs:main1
442761992115,4cyclictest660762-21apt-get10:40:120
442761992115,4cyclictest335557-21systemd-journal09:30:000
442772992017,2cyclictest712347-21kworker/u8:1+flush-259:011:50:203
442772992017,2cyclictest712347-21kworker/u8:1+events_unbound12:35:213
442772992017,2cyclictest691114-21kworker/u8:2+flush-259:011:20:163
442772992016,3cyclictest691114-21kworker/u8:2+flush-259:011:15:283
442772992016,3cyclictest660538-21kworker/u8:0+events_unbound10:51:303
442772992016,3cyclictest650357-21kworker/u8:3+events_unbound11:40:273
442772992016,3cyclictest582704-21kworker/u8:3+events_unbound09:52:303
442772992014,4cyclictest335557-21systemd-journal09:23:283
44277099203,9cyclictest629085-21munin-run10:10:002
44277099201,17cyclictest675946-21awk10:55:002
442766992017,2cyclictest531183-21kworker/u8:3+events_unbound08:43:281
442766992017,2cyclictest510809-21kworker/u8:2+flush-259:008:20:111
442766992016,3cyclictest712347-21kworker/u8:1+events_unbound11:50:131
442766992016,3cyclictest660538-21kworker/u8:0+events_unbound10:46:291
442766992014,4cyclictest650357-21kworker/u8:3+events_unbound11:40:231
442766992013,5cyclictest776609-21cp12:30:011
442761992015,4cyclictest448149-21apt-get07:15:110
442761992013,5cyclictest691184-21apt-get11:06:380
442772991916,2cyclictest712347-21kworker/u8:1+flush-259:012:30:213
442772991916,2cyclictest499283-21kworker/u8:1+events_unbound09:05:193
442772991915,3cyclictest582704-21kworker/u8:3+flush-259:009:35:303
442772991915,3cyclictest473836-21kworker/u8:4+events_unbound07:49:193
442772991915,3cyclictest458212-21kworker/u8:3+events_unbound08:08:273
44277099198,9cyclictest0-21swapper/209:50:002
44277099198,9cyclictest0-21swapper/209:50:002
442770991914,3cyclictest691184-21apt-get11:06:422
44277099191,16cyclictest712623-21latency_hist11:30:002
442766991916,2cyclictest561629-21kworker/u8:2+flush-259:009:10:191
442766991915,3cyclictest765859-21kworker/u8:2+flush-259:012:20:201
442766991915,3cyclictest628696-21kworker/u8:2+events_unbound10:10:221
442766991915,3cyclictest561629-21kworker/u8:2+events_unbound09:25:071
442766991915,3cyclictest458212-21kworker/u8:3+events_unbound08:05:001
442772991815,3cyclictest499283-21kworker/u8:1+events_unbound08:45:013
442772991815,2cyclictest738345-21kworker/u8:0+events_unbound11:58:293
442772991815,2cyclictest644511-21kworker/u8:1+events_unbound10:33:293
442772991815,2cyclictest499283-21kworker/u8:1+events_unbound08:50:223
442772991814,3cyclictest628696-21kworker/u8:2+events_unbound10:23:303
442772991814,3cyclictest603480-21kworker/u8:1+flush-259:010:00:313
442772991814,3cyclictest442682-21kworker/u8:0+events_unbound08:00:203
442772991813,4cyclictest335557-21systemd-journal09:25:283
442770991814,3cyclictest744585-21gpgv12:00:002
442770991814,2cyclictest0-21swapper/211:05:012
442770991812,4cyclictest639662-21/usr/sbin/munin10:20:222
44277099181,15cyclictest613601-21idleruntime-cro09:55:002
44277099180,16cyclictest1-21systemd08:55:012
44277099180,16cyclictest0-21swapper/209:00:172
44277099180,15cyclictest623937-21latency_hist10:05:002
442766991815,3cyclictest442682-21kworker/u8:0+events_unbound07:59:581
442766991815,2cyclictest702539-21kworker/u8:0+flush-259:011:20:291
442766991815,2cyclictest458212-21kworker/u8:3+events_unbound08:05:211
442766991814,3cyclictest712347-21kworker/u8:1+flush-259:012:05:251
442766991814,3cyclictest427569-21kworker/u8:3+events_unbound07:10:301
442766991813,4cyclictest499283-21kworker/u8:1+events_unbound08:45:121
442761991813,3cyclictest582667-21latency_hist09:25:000
442772991713,3cyclictest712347-21kworker/u8:1+events_unbound12:30:003
442772991713,3cyclictest442682-21kworker/u8:0+flush-259:007:20:253
442772991713,3cyclictest442682-21kworker/u8:0+flush-259:007:20:253
442772991711,4cyclictest520436-21apt-get08:25:013
44277099177,9cyclictest782084-21fschecks_time12:35:132
44277099171,13cyclictest603222-21latency_hist09:45:012
44277099170,9cyclictest665341-21taskset10:41:352
44277099170,9cyclictest509628-21taskset08:14:232
44277099170,15cyclictest592444-21tail09:30:252
44277099170,14cyclictest479030-21cat07:45:012
442766991713,3cyclictest660538-21kworker/u8:0+events_unbound10:50:211
442766991711,4cyclictest723045-21cp11:40:001
442761991714,2cyclictest692663-21irqcore11:10:160
442761991714,2cyclictest60350irq/132-enp0s29f1:rx-009:34:130
442761991714,2cyclictest60350irq/132-enp0s29f1:rx-007:28:110
442761991713,3cyclictest603480-21kworker/u8:1+events_unbound10:05:240
442761991713,2cyclictest0-21swapper/012:40:010
442761991711,4cyclictest722939-21awk11:40:000
442761991711,4cyclictest571423-21ntpq09:10:210
442772991613,2cyclictest765859-21kworker/u8:2+flush-259:012:20:153
442772991613,2cyclictest410131-21kworker/u8:2+flush-259:007:25:203
442772991612,3cyclictest660538-21kworker/u8:0+events_unbound10:58:293
442772991612,3cyclictest561629-21kworker/u8:2+events_unbound09:31:303
442772991612,3cyclictest536951-21kworker/u8:0+flush-259:008:40:213
442772991612,3cyclictest458212-21kworker/u8:3+flush-259:007:50:153
44277099164,10cyclictest760500-21cat12:15:002
44277099161,13cyclictest755288-21cut12:10:002
44277099161,13cyclictest755288-21cut12:10:002
44277099161,13cyclictest728746-21gpgconf11:45:002
44277099161,13cyclictest587795-21turbostat.cron09:29:592
44277099161,13cyclictest556852-21cron09:00:002
44277099161,13cyclictest536976-21irqcore08:40:162
44277099161,13cyclictest531158-21turbostat.cron08:35:002
44277099161,13cyclictest499393-21munin-plugin-st08:05:002
44277099161,12cyclictest520791-21apt-key08:25:012
44277099160,14cyclictest582663-21sh09:25:002
44277099160,14cyclictest567177-21idleruntime-cro09:10:002
44277099160,14cyclictest500071-21sort08:05:192
44277099160,14cyclictest485234-21grep07:50:162
44277099160,14cyclictest483967-21awk07:50:002
44277099160,14cyclictest468483-21basename07:35:002
44277099160,13cyclictest760827-21tr12:15:132
44277099160,13cyclictest734527-21tr11:50:142
44277099160,13cyclictest458186-21latency_hist07:25:012
44277099160,13cyclictest458186-21latency_hist07:25:002
442766991613,2cyclictest410131-21kworker/u8:2+events_unbound07:25:181
442766991610,4cyclictest582704-21kworker/u8:3+events_unbound10:05:541
442761991613,2cyclictest60350irq/132-enp0s29f1:rx-012:22:330
442761991613,2cyclictest60350irq/132-enp0s29f1:rx-012:16:400
442761991613,2cyclictest60350irq/132-enp0s29f1:rx-010:55:160
442761991613,2cyclictest60350irq/132-enp0s29f1:rx-009:59:420
442761991613,2cyclictest0-21swapper/007:55:000
442761991612,3cyclictest755225-21kworker/u8:3+flush-259:012:10:250
442761991612,3cyclictest60350irq/132-enp0s29f1:rx-010:10:210
442761991612,3cyclictest582704-21kworker/u8:3+flush-259:009:50:230
442761991612,3cyclictest562690-21grep09:05:150
442761991612,3cyclictest499283-21kworker/u8:1+flush-259:008:40:200
442772991512,3cyclictest582704-21kworker/u8:3+events_unbound09:44:373
442772991511,3cyclictest702539-21kworker/u8:0+flush-259:011:25:283
442772991511,3cyclictest442682-21kworker/u8:0+events_unbound08:10:173
44277099151,13cyclictest507-21in:imuxsock09:40:002
44277099151,12cyclictest650552-21diskstats10:30:132
44277099151,12cyclictest545774-21ntpq08:45:222
44277099151,11cyclictest713044-21df_inode11:30:122
44277099151,11cyclictest666043-21grep10:45:132
44277099150,13cyclictest702283-21cron11:19:592
44277099150,13cyclictest577482-21cron09:20:002
44277099150,13cyclictest572442-21wc09:14:592
44277099150,13cyclictest525043-21tail08:25:202
44277099150,13cyclictest469169-21ps07:35:152
44277099150,13cyclictest458731-21perl07:25:142
44277099150,13cyclictest448258-21cpuspeed_turbos07:15:122
44277099150,12cyclictest729340-21tr11:45:122
44277099150,12cyclictest702819-21fschecks_time11:20:142
44277099150,12cyclictest634615-21cpuspeed_turbos10:15:112
44277099150,12cyclictest629436-21tr10:10:132
44277099150,12cyclictest489432-21tr07:55:122
442766991511,3cyclictest448555-21kworker/u8:1+flush-259:007:35:301
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-010:45:150
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-010:35:170
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-009:40:230
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-009:16:450
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-008:55:030
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-008:30:190
442761991513,1cyclictest60350irq/132-enp0s29f1:rx-007:35:190
442761991512,2cyclictest776919-21fschecks_time12:30:140
442761991512,2cyclictest60350irq/132-enp0s29f1:rx-011:40:130
442761991512,2cyclictest60350irq/132-enp0s29f1:rx-011:03:570
442761991512,2cyclictest60350irq/132-enp0s29f1:rx-007:32:540
442761991512,2cyclictest499382-21sh08:05:010
442761991511,2cyclictest60350irq/132-enp0s29f1:rx-008:50:210
442761991510,4cyclictest442682-21kworker/u8:0+events_unbound07:45:240
44277299149,4cyclictest472694-21ntpq07:35:203
44277099141,11cyclictest617901-21/usr/sbin/munin09:55:202
44277099141,11cyclictest535817-21grep08:35:262
44277099140,12cyclictest766034-21cpuspeed_turbos12:20:132
44277099140,12cyclictest655984-21ps10:35:182
442761991412,1cyclictest60350irq/132-enp0s29f1:rx-012:25:210
442761991412,1cyclictest60350irq/132-enp0s29f1:rx-011:25:430
442761991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:220
442761991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:220
442761991411,2cyclictest60350irq/132-enp0s29f1:rx-010:31:460
442761991411,2cyclictest60350irq/132-enp0s29f1:rx-007:40:150
442761991411,2cyclictest442682-21kworker/u8:0+flush-259:007:20:260
442761991411,2cyclictest442682-21kworker/u8:0+flush-259:007:20:250
442761991410,3cyclictest749162-21grep12:00:210
442761991410,3cyclictest644801-21/usr/sbin/munin10:25:130
44277299139,3cyclictest510809-21kworker/u8:2+flush-259:008:30:203
44277299139,2cyclictest0-21swapper/309:04:593
442772991310,2cyclictest458212-21kworker/u8:3+flush-259:008:20:113
442772991310,2cyclictest427569-21kworker/u8:3+flush-259:007:11:023
442772991310,1cyclictest0-21swapper/308:35:263
442761991311,1cyclictest60350irq/132-enp0s29f1:rx-009:03:390
442761991310,2cyclictest60350irq/132-enp0s29f1:rx-011:55:100
44276199129,2cyclictest697624-21fschecks_count11:15:150
442761991210,1cyclictest60350irq/132-enp0s29f1:rx-010:50:250
442761991210,1cyclictest60350irq/132-enp0s29f1:rx-010:20:160
44277299118,2cyclictest458212-21kworker/u8:3+flush-259:007:40:243
44277299118,2cyclictest0-21swapper/311:30:163
44276199118,1cyclictest0-21swapper/012:05:180
44276199118,1cyclictest0-21swapper/012:05:180
44276199117,2cyclictest60350irq/132-enp0s29f1:rx-008:10:130
4427729997,1cyclictest481rcuc/310:10:173
4427729996,2cyclictest571905-21unixbench_singl09:10:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional