You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 21:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Mon May 25, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
311542239163,24sleep20-21swapper/207:06:322
313172237161,24sleep00-21swapper/007:08:390
312032203168,23sleep10-21swapper/107:07:111
311282202165,25sleep30-21swapper/307:06:133
1799421580,4sleep33152499cyclictest11:30:163
1799421580,4sleep33152499cyclictest11:30:163
315209910198,2cyclictest0-21swapper/208:26:562
315209910098,1cyclictest0-21swapper/208:18:562
190242870,5sleep23152099cyclictest07:50:272
131722610,4sleep013174-21cat07:40:010
31520995451,2cyclictest0-21swapper/207:12:432
201052540,4sleep230-21rcuc/210:20:242
201052540,4sleep230-21rcuc/210:20:242
167442530,5sleep13151399cyclictest09:00:161
3150799366,14cyclictest18016-21munin-run09:05:010
3150799348,13cyclictest28998-21munin-run09:30:010
3152099311,6cyclictest0-21swapper/210:26:282
3151399312,23cyclictest20440-21kworker/u8:109:14:041
3151399312,23cyclictest20440-21kworker/u8:109:14:041
3150799314,12cyclictest4219-21munin-run07:20:010
3152499291,13cyclictest0-21swapper/308:54:493
3151399291,13cyclictest0-21swapper/111:39:161
3151399291,11cyclictest0-21swapper/109:20:581
31507992910,16cyclictest15460-21munin-run12:40:010
3152499285,7cyclictest0-21swapper/309:39:013
31524992818,3cyclictest0-21swapper/309:27:233
3152499281,14cyclictest0-21swapper/307:18:213
3151399281,11cyclictest12643-21sh11:20:011
3150799281,5cyclictest0-21swapper/010:05:240
3152499271,12cyclictest0-21swapper/310:52:373
3151399273,19cyclictest25698-40dnf11:48:251
3151399272,12cyclictest0-21swapper/111:03:571
31513992715,3cyclictest0-21swapper/108:08:061
3151399271,20cyclictest0-21swapper/110:32:311
3151399271,11cyclictest8925-21grep07:30:121
3151399271,10cyclictest0-21swapper/112:29:071
3150799270,12cyclictest7032-21basename08:40:010
3152499263,15cyclictest0-21swapper/312:29:343
3152499261,17cyclictest0-21swapper/311:56:113
3152099261,5cyclictest7430-21kworker/u8:110:05:132
3152099261,5cyclictest0-21swapper/211:29:252
31513992619,3cyclictest0-21swapper/112:21:261
31513992619,3cyclictest0-21swapper/107:54:281
31513992614,3cyclictest0-21swapper/110:41:341
31513992613,3cyclictest0-21swapper/112:01:241
31513992613,3cyclictest0-21swapper/108:22:401
31513992613,3cyclictest0-21swapper/107:48:261
31513992613,3cyclictest0-21swapper/107:13:501
3151399261,22cyclictest29939-21grep10:45:121
3151399261,19cyclictest0-21swapper/109:52:551
3151399261,14cyclictest0-21swapper/110:25:071
31513992610,3cyclictest28098-21grep11:55:011
3151399260,20cyclictest0-21swapper/108:47:131
3152499254,12cyclictest17065-21kworker/u8:111:45:053
3152499251,6cyclictest25730-21expr10:35:123
3152099251,6cyclictest0-21swapper/211:06:562
3152099251,4cyclictest1355-21kworker/u8:011:14:362
3151399252,11cyclictest28597-21df11:55:121
31513992518,3cyclictest0-21swapper/110:01:041
31513992513,3cyclictest0-21swapper/112:08:251
31513992513,3cyclictest0-21swapper/111:05:031
31513992513,3cyclictest0-21swapper/107:24:041
31513992513,2cyclictest0-21swapper/112:39:251
31513992512,3cyclictest0-21swapper/109:37:261
31513992512,3cyclictest0-21swapper/109:26:061
31513992512,3cyclictest0-21swapper/107:55:301
3151399251,21cyclictest7020-21awk08:40:011
3151399251,19cyclictest0-21swapper/110:09:311
3151399251,10cyclictest0-21swapper/110:21:291
3151399251,10cyclictest0-21swapper/110:21:291
3151399251,10cyclictest0-21swapper/110:14:501
3151399250,19cyclictest0-21swapper/109:57:051
3150799259,14cyclictest0-21swapper/008:35:010
3150799251,5cyclictest0-21swapper/009:31:210
31507992514,3cyclictest8493-21users12:20:260
208482251,3sleep320882-21ntp_states07:55:183
3152499249,6cyclictest23909-21swap09:15:253
3152499246,3cyclictest14522-21sshd08:55:163
3152499242,4cyclictest0-21swapper/310:16:473
3152099241,9cyclictest0-21swapper/208:58:052
3152099241,4cyclictest0-21swapper/209:47:232
31513992421,2cyclictest0-21swapper/112:14:531
31513992421,2cyclictest0-21swapper/108:56:441
3151399241,6cyclictest0-21swapper/110:16:561
31513992416,3cyclictest0-21swapper/109:30:401
31513992412,3cyclictest0-21swapper/110:55:081
31513992412,3cyclictest0-21swapper/108:00:051
31513992412,3cyclictest0-21swapper/108:00:051
3151399241,21cyclictest13005-21awk07:40:011
3151399241,21cyclictest0-21swapper/109:05:131
31513992412,1cyclictest0-21swapper/107:26:421
3151399241,19cyclictest0-21swapper/108:26:201
3151399241,18cyclictest0-21swapper/110:53:341
3151399241,11cyclictest0-21swapper/111:26:401
3151399241,11cyclictest0-21swapper/109:18:161
3151399241,10cyclictest0-21swapper/109:45:091
3150799241,5cyclictest0-21swapper/011:33:420
3150799241,5cyclictest0-21swapper/011:33:420
3152499239,7cyclictest0-21swapper/312:12:233
3152499235,3cyclictest0-21swapper/310:23:033
3152499235,3cyclictest0-21swapper/310:23:033
3152499235,10cyclictest0-21swapper/309:49:583
3152499231,9cyclictest0-21swapper/308:44:173
3152499231,4cyclictest0-21swapper/311:19:513
31524992311,3cyclictest0-21swapper/312:18:273
3152499231,11cyclictest1-21systemd11:40:163
3152499231,10cyclictest0-21swapper/307:27:373
3152499230,5cyclictest25172-21kworker/u8:210:34:533
3152499230,15cyclictest24266-21kworker/u8:009:21:303
3152499230,11cyclictest0-21swapper/309:52:493
3152099235,4cyclictest579-21systemd-journal12:30:012
3151399232,18cyclictest1595-21df09:40:121
31513992320,2cyclictest0-21swapper/111:11:381
31513992320,2cyclictest0-21swapper/107:43:391
31513992316,2cyclictest0-21swapper/108:11:411
3151399231,4cyclictest0-21swapper/112:34:591
31513992313,2cyclictest0-21swapper/111:21:491
31513992312,2cyclictest0-21swapper/111:31:421
31513992312,2cyclictest0-21swapper/111:31:421
3151399231,18cyclictest0-21swapper/108:53:181
3151399231,11cyclictest9252-21cat08:45:021
3151399231,11cyclictest5709-21netstat12:15:191
3150799235,5cyclictest2755-21grep09:40:250
3150799231,7cyclictest0-21swapper/011:18:290
3150799231,6cyclictest0-21swapper/011:02:020
3150799231,6cyclictest0-21swapper/010:50:100
3150799231,5cyclictest0-21swapper/011:05:120
3150799231,4cyclictest0-21swapper/010:28:140
3150799231,11cyclictest0-21swapper/008:20:010
31507992310,6cyclictest0-21swapper/011:57:160
241692230,5sleep230-21rcuc/210:30:192
3152499228,7cyclictest0-21swapper/311:23:583
3152499227,8cyclictest0-21swapper/309:14:153
3152499227,8cyclictest0-21swapper/309:14:153
3152499223,3cyclictest0-21swapper/310:02:473
3152499222,4cyclictest0-21swapper/312:33:143
3152499222,13cyclictest87350irq/126-enp1s0-09:30:133
3152499221,16cyclictest0-21swapper/311:26:033
3152499221,14cyclictest0-21swapper/311:11:573
3152499221,12cyclictest31707-21/usr/sbin/munin07:10:223
3152499221,12cyclictest28243-21tail10:40:163
3152499221,12cyclictest26065-21/usr/sbin/munin11:50:253
3152499220,10cyclictest0-21swapper/311:07:293
3152099221,7cyclictest4084-21/usr/sbin/munin11:00:232
3152099221,7cyclictest29325-21ntp_states11:55:202
3152099221,6cyclictest29606-21latency_hist10:45:022
3152099221,6cyclictest1015-21latency_hist09:40:012
31520992216,3cyclictest23275-21sed09:15:172
3152099221,4cyclictest0-21swapper/209:25:282
3152099221,2cyclictest28635-21kworker/u8:209:52:242
3151399227,4cyclictest0-21swapper/110:39:251
3151399221,10cyclictest3777-21grep08:30:191
3151399221,10cyclictest0-21swapper/111:41:091
3151399220,14cyclictest0-21swapper/108:15:111
3151399220,10cyclictest0-21swapper/107:15:251
3150799224,5cyclictest579-21systemd-journal09:20:160
3150799223,10cyclictest11-21rcuc/011:35:140
3150799222,6cyclictest22160-21tune2fs11:40:150
3150799221,5cyclictest0-21swapper/008:10:260
3150799221,15cyclictest0-21swapper/010:13:480
243562221,2sleep024539-21/usr/sbin/munin11:45:150
3152499215,9cyclictest0-21swapper/312:38:273
3152499211,9cyclictest31063-21head12:00:153
3152499211,8cyclictest17107-21ntp_states09:00:213
3152499211,6cyclictest0-21swapper/308:25:203
3152499211,3cyclictest0-21swapper/310:10:213
3152499211,2cyclictest87350irq/126-enp1s0-11:35:253
3152499211,13cyclictest0-21swapper/312:06:313
31524992110,3cyclictest0-21swapper/310:08:533
3152099214,5cyclictest20217-21diskmemload09:21:222
3152099214,4cyclictest27094-21hddtemp_smartct08:10:162
3152099212,5cyclictest8599-21latency_hist07:30:022
3152099212,5cyclictest11750-21ntp_states11:15:212
3152099211,7cyclictest0-21swapper/212:07:482
3152099211,7cyclictest0-21swapper/211:47:412
3152099211,6cyclictest6673-21latency_hist12:20:012
3152099211,6cyclictest4939-21cat08:35:022
3152099211,6cyclictest0-21swapper/212:02:302
3152099211,5cyclictest2797-21perl07:15:152
3150799219,2cyclictest0-21swapper/009:50:270
3150799218,2cyclictest21297-21sed07:55:220
3150799218,2cyclictest0-21swapper/009:10:240
3150799218,2cyclictest0-21swapper/009:10:240
3150799217,3cyclictest24655-1kworker/u9:010:41:050
3150799216,3cyclictest87350irq/126-enp1s0-07:30:240
3150799215,2cyclictest19585-21timerandwakeup09:05:240
3150799214,9cyclictest7-21ksoftirqd/011:10:240
3150799214,9cyclictest11-21rcuc/012:05:240
3150799214,3cyclictest31084-21sshd12:00:160
3150799213,10cyclictest14025-21pmu-power11:20:230
3150799213,10cyclictest11-21rcuc/009:35:240
3150799211,9cyclictest0-21swapper/008:25:270
3150799211,7cyclictest0-21swapper/011:50:240
3150799211,6cyclictest14215-21needreboot07:40:190
3150799211,6cyclictest0-21swapper/010:01:070
3150799211,5cyclictest5969-21smart_sda12:15:240
3150799211,5cyclictest0-21swapper/012:30:120
31507992115,2cyclictest0-21swapper/010:30:220
3150799211,17cyclictest11404-21idleruntime-cro08:50:010
3150799211,12cyclictest808-21snmpd08:42:480
3152499207,7cyclictest0-21swapper/310:25:053
3152499203,2cyclictest0-21swapper/311:01:343
3152499202,10cyclictest2851-21/usr/sbin/munin08:30:223
3152499201,6cyclictest20217-21diskmemload11:00:023
3152499201,6cyclictest0-21swapper/312:24:023
3152499201,4cyclictest0-21swapper/310:47:163
31524992013,2cyclictest0-21swapper/308:00:133
31524992013,2cyclictest0-21swapper/308:00:133
31524992011,3cyclictest0-21swapper/309:55:103
3152499201,11cyclictest24223-21/usr/sbin/munin08:05:123
3152499201,11cyclictest13222-21/usr/sbin/munin07:40:163
3152499201,10cyclictest30806-21/usr/sbin/munin08:20:123
31524992010,3cyclictest0-21swapper/309:40:033
3152099207,2cyclictest8995-21pmu-power09:55:222
3152099204,4cyclictest22013-21/usr/sbin/munin08:00:282
3152099204,4cyclictest22013-21/usr/sbin/munin08:00:282
3152099204,3cyclictest31672-40dnf10:48:162
3152099203,4cyclictest13001-21munin-run07:40:012
3152099202,6cyclictest27408-21sh10:40:012
3152099202,3cyclictest16036-21/usr/sbin/munin09:00:182
3152099201,9cyclictest0-21swapper/210:50:192
3152099201,9cyclictest0-21swapper/209:40:282
3152099201,7cyclictest13292-21tune2fs11:20:152
3152099201,6cyclictest22530-21cat09:15:012
3152099201,6cyclictest22530-21cat09:15:012
3152099201,5cyclictest9437-21/usr/sbin/munin08:45:152
3152099201,5cyclictest5395-21diskstats08:35:152
3152099201,5cyclictest31154-21diskstats08:20:112
3152099201,5cyclictest19987-21tune2fs11:35:142
3152099201,5cyclictest18557-21sshd10:19:162
3152099201,5cyclictest0-21swapper/211:51:512
3152099201,4cyclictest0-21swapper/210:04:232
3150799208,5cyclictest24655-1kworker/u9:010:39:240
3150799201,6cyclictest11-21rcuc/010:55:160
3150799201,4cyclictest26197-21munin-run08:10:010
31507992011,2cyclictest30439-21grep10:45:150
3150799200,9cyclictest0-21swapper/009:55:130
3150799200,9cyclictest0-21swapper/009:47:590
3152499196,3cyclictest0-21swapper/308:36:163
3152499196,2cyclictest10808-21awk07:35:013
3152499193,4cyclictest0-21swapper/307:45:183
3152499191,7cyclictest0-21swapper/307:54:133
3152499191,6cyclictest11705-21smartctl07:35:163
3152499191,10cyclictest808-21snmpd08:17:413
31524991910,3cyclictest808-21snmpd07:22:323
3152099193,8cyclictest87350irq/126-enp1s0-07:30:162
3152099193,7cyclictest87650irq/129-enp1s0-12:20:162
3152099191,9cyclictest0-21swapper/212:39:052
3152099191,7cyclictest0-21swapper/210:57:532
3152099191,6cyclictest16637-21cat10:15:012
3152099191,5cyclictest18242-21/usr/sbin/munin09:05:162
3152099191,4cyclictest22733-21grep11:40:212
3152099191,4cyclictest15191-21latency_hist07:45:012
31520991914,3cyclictest11704-21cpuspeed_turbos08:50:132
3152099191,11cyclictest0-21swapper/212:30:152
3150799197,2cyclictest0-21swapper/007:47:370
3150799195,2cyclictest9-21rcu_preempt07:50:140
3150799194,3cyclictest87550irq/128-enp1s0-10:15:230
3150799192,3cyclictest12050irq/123-ahci[0007:15:010
3150799191,6cyclictest15066-21gzip08:55:230
3150799191,6cyclictest12011-21fschecks_count08:50:160
3150799191,5cyclictest0-21swapper/012:14:320
3150799191,4cyclictest0-21swapper/012:29:260
3150799191,4cyclictest0-21swapper/010:20:140
3150799191,4cyclictest0-21swapper/010:20:140
3150799191,4cyclictest0-21swapper/009:17:200
3150799191,2cyclictest87350irq/126-enp1s0-08:00:240
3150799191,2cyclictest87350irq/126-enp1s0-08:00:240
3150799191,16cyclictest5431-21strings07:20:190
3150799190,9cyclictest0-21swapper/011:25:210
3152499189,2cyclictest0-21swapper/308:10:243
3152499180,4cyclictest0-21swapper/308:45:143
3152099181,7cyclictest790-21in:imjournal08:41:202
3152099181,7cyclictest0-21swapper/212:11:152
3152099181,6cyclictest24933-21idleruntime08:05:162
3152099181,6cyclictest17273-21/usr/sbin/munin11:30:222
3152099181,6cyclictest17273-21/usr/sbin/munin11:30:222
3152099181,5cyclictest0-21swapper/209:30:072
3150799180,3cyclictest5750irq/9-acpi08:23:580
31524991710,4cyclictest20207-21crond09:10:013
3152099171,5cyclictest19911-21sort07:55:092
3152099171,3cyclictest4439-21/usr/sbin/munin07:20:242
3150799171,10cyclictest0-21swapper/007:25:270
3152099161,14cyclictest0-21swapper/207:46:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional