You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 12:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Aug 04, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
74912236163,23sleep20-21swapper/219:08:572
74072236163,22sleep10-21swapper/119:07:501
72402236162,25sleep00-21swapper/019:05:400
73322203167,24sleep30-21swapper/319:06:523
89192570,4sleep011-21rcuc/022:50:190
167702520,3sleep341-21ksoftirqd/321:55:153
101982490,1sleep30-21swapper/319:15:023
185252450,4sleep121-21rcuc/119:30:221
51972440,1sleep20-21swapper/222:40:252
301882400,1sleep30-21swapper/322:25:183
766899381,30cyclictest0-21swapper/119:37:391
766399377,13cyclictest17121-21munin-run19:30:000
767999342,20cyclictest0-21swapper/321:28:243
766399348,12cyclictest4555-21idleruntime-cro20:15:010
766899310,24cyclictest0-21swapper/122:44:431
766399311,9cyclictest15764-21/usr/sbin/munin20:40:240
766899301,22cyclictest0-21swapper/123:47:441
766899301,11cyclictest0-21swapper/120:38:191
766899300,19cyclictest0-21swapper/121:01:281
766899300,11cyclictest0-21swapper/123:50:181
766899291,12cyclictest0-21swapper/122:38:591
766899290,8cyclictest0-21swapper/120:22:511
766899290,22cyclictest0-21swapper/121:48:281
767999281,14cyclictest0-21swapper/321:35:453
767399281,11cyclictest20164-21/usr/sbin/munin20:50:172
766899281,9cyclictest0-21swapper/121:30:161
766899281,7cyclictest0-21swapper/100:16:321
766899281,12cyclictest0-21swapper/100:24:061
766899281,11cyclictest0-21swapper/120:32:251
766899281,10cyclictest0-21swapper/120:26:391
766899281,10cyclictest0-21swapper/119:27:421
766399282,14cyclictest5164-21munin-run21:30:010
61682280,9sleep341-21ksoftirqd/322:45:153
766899273,13cyclictest19614-40dnf00:30:031
766899271,11cyclictest0-21swapper/122:10:471
766899270,11cyclictest0-21swapper/123:24:481
766899270,11cyclictest0-21swapper/121:56:321
766899270,11cyclictest0-21swapper/121:05:161
766899270,11cyclictest0-21swapper/120:19:071
766399270,14cyclictest18966-21idleruntime-cro23:15:020
766399270,13cyclictest26547-21idleruntime-cro21:05:020
75972270,3sleep1766899cyclictest22:45:301
766899261,9cyclictest0-21swapper/120:52:231
766899261,9cyclictest0-21swapper/119:10:151
766899261,9cyclictest0-21swapper/100:09:481
766899261,5cyclictest4088-21sed21:25:201
766899261,11cyclictest0-21swapper/120:08:051
766899261,10cyclictest0-21swapper/121:51:081
766899260,21cyclictest0-21swapper/120:13:391
766899260,17cyclictest0-21swapper/100:37:391
766899260,11cyclictest0-21swapper/122:55:511
766899260,10cyclictest0-21swapper/123:12:511
766399261,11cyclictest1731-21kworker/0:023:58:530
767399258,6cyclictest7556-21cat21:35:022
766899252,3cyclictest17733-21munin-run20:45:001
766899251,9cyclictest0-21swapper/123:05:151
7668992514,3cyclictest27559-21cstates22:20:111
766899251,12cyclictest0-21swapper/100:11:321
766899251,11cyclictest0-21swapper/121:35:321
766899251,10cyclictest0-21swapper/123:30:161
766899251,10cyclictest0-21swapper/119:41:291
766899250,8cyclictest0-21swapper/119:50:031
766899250,5cyclictest0-21swapper/123:03:071
766899250,16cyclictest0-21swapper/122:17:471
766899250,10cyclictest0-21swapper/100:03:561
766399258,14cyclictest9573-21perf21:40:000
211962250,2sleep10-21swapper/122:05:161
211962250,2sleep10-21swapper/122:05:161
767999245,10cyclictest26843-21idleruntime19:50:173
767399244,7cyclictest5019-40dnf21:28:242
766899245,3cyclictest0-21swapper/100:26:121
766899245,3cyclictest0-21swapper/100:26:121
766899241,9cyclictest0-21swapper/122:52:351
766899241,9cyclictest0-21swapper/121:12:201
766899241,12cyclictest0-21swapper/123:37:001
766899241,11cyclictest808-21snmpd19:46:131
766899241,11cyclictest808-21snmpd19:46:131
766899241,10cyclictest0-21swapper/119:57:271
766899241,10cyclictest0-21swapper/119:16:091
766899240,9cyclictest0-21swapper/122:27:431
766899240,9cyclictest0-21swapper/122:01:431
766899240,9cyclictest0-21swapper/119:23:001
153152240,8sleep07-21ksoftirqd/021:50:220
89982230,1sleep30-21swapper/320:25:023
767999233,4cyclictest0-21swapper/322:20:143
766899237,3cyclictest0-21swapper/121:21:101
766899231,9cyclictest0-21swapper/123:17:271
766899231,9cyclictest0-21swapper/123:17:271
766899230,9cyclictest0-21swapper/121:16:081
766899230,8cyclictest0-21swapper/123:57:201
766399231,6cyclictest0-21swapper/000:01:220
767999227,4cyclictest0-21swapper/322:30:143
767999221,12cyclictest0-21swapper/323:15:233
767999221,12cyclictest0-21swapper/323:15:233
766899225,3cyclictest0-21swapper/120:59:041
766899225,3cyclictest0-21swapper/120:59:041
766899221,18cyclictest25142-21awk23:25:271
766899221,16cyclictest0-21swapper/122:31:281
766399223,11cyclictest26759-21/usr/sbin/munin21:05:240
7663992218,3cyclictest7-21ksoftirqd/023:40:010
766399221,4cyclictest0-21swapper/000:28:010
766399221,4cyclictest0-21swapper/000:28:010
766399221,12cyclictest5389-21/usr/sbin/munin21:30:130
295962220,5sleep229599-21gltestperf21:10:152
767999218,3cyclictest0-21swapper/300:00:273
767999215,6cyclictest0-21swapper/323:04:543
7679992115,4cyclictest1680-21sed21:20:163
767999211,11cyclictest0-21swapper/322:10:493
767999211,10cyclictest10784-21needreboot21:40:213
767399216,4cyclictest0-21swapper/223:47:402
766899217,3cyclictest0-21swapper/120:49:351
766899211,9cyclictest0-21swapper/121:42:401
766899210,9cyclictest0-21swapper/123:41:161
766899210,9cyclictest0-21swapper/120:01:471
766399212,9cyclictest13113-21kworker/0:220:37:420
766399211,4cyclictest7580-21kworker/u8:220:24:060
766399211,4cyclictest0-21swapper/000:21:280
7663992114,4cyclictest7-21ksoftirqd/022:50:010
7663992114,4cyclictest12085-21ntpq00:10:190
248342211,2sleep20-21swapper/223:25:252
767999209,3cyclictest11047-21/usr/sbin/munin00:10:233
767999208,4cyclictest20791-21/usr/sbin/munin22:05:273
767999208,4cyclictest20791-21/usr/sbin/munin22:05:273
767999207,7cyclictest87350irq/126-enp1s0-23:36:423
767999202,10cyclictest17637-21/usr/sbin/munin00:25:123
767999202,10cyclictest17637-21/usr/sbin/munin00:25:123
767999201,9cyclictest16704-21grep19:25:253
767999201,8cyclictest0-21swapper/321:31:433
7679992014,3cyclictest12596-21grep21:45:183
7679992011,6cyclictest808-21snmpd20:42:003
767999201,11cyclictest3295-21ntp_states23:50:213
767999201,11cyclictest23271-21memory20:55:193
767999201,11cyclictest23271-21memory20:55:193
767999201,11cyclictest0-21swapper/323:20:263
767999201,11cyclictest0-21swapper/322:52:553
767999201,10cyclictest32145-21grep21:15:213
767999201,10cyclictest26442-21smart_sda22:15:243
767999201,10cyclictest22035-21/usr/sbin/munin00:35:163
767999201,10cyclictest14930-21tr23:05:123
767999201,10cyclictest0-21swapper/320:30:103
767399206,3cyclictest0-21swapper/223:16:252
767399206,3cyclictest0-21swapper/223:16:252
767399201,5cyclictest9144-21cat00:05:132
767399201,4cyclictest0-21swapper/200:01:112
767399201,2cyclictest87350irq/126-enp1s0-20:00:582
767399200,4cyclictest6079-21kworker/u8:023:57:202
766399203,5cyclictest755-21rngd00:17:320
7663992014,3cyclictest22060-21gltestperf23:20:170
766399201,16cyclictest11758-21munin-run21:45:000
767999195,6cyclictest39-21rcuc/300:05:133
767999194,4cyclictest0-21swapper/323:32:133
767999193,7cyclictest0-21swapper/323:25:413
767999193,6cyclictest0-21swapper/321:12:233
767999193,14cyclictest32314-21kworker/3:219:20:243
767999191,15cyclictest5136-21switchtime22:40:253
767999191,11cyclictest0-21swapper/322:59:263
767999191,10cyclictest4422-21/usr/sbin/munin23:55:233
767999191,10cyclictest20164-21/usr/sbin/munin20:50:183
767399198,4cyclictest7995-21awk22:50:012
767399194,3cyclictest0-21swapper/200:22:512
767399191,6cyclictest0-21swapper/223:11:572
767399191,6cyclictest0-21swapper/200:39:282
767399191,5cyclictest0-21swapper/200:18:292
767399191,3cyclictest0-21swapper/220:40:142
7673991913,3cyclictest18912-21head19:30:262
7673991913,3cyclictest16723-21df_inode21:55:142
767399190,13cyclictest0-21swapper/221:36:312
766399193,4cyclictest0-21swapper/022:35:130
7663991914,3cyclictest22131-21munin-run20:55:020
766399191,13cyclictest0-21swapper/019:35:200
766399191,10cyclictest18581-21/usr/sbin/munin22:00:200
766399191,10cyclictest12602-21/usr/sbin/munin23:00:280
766399191,10cyclictest10758-21/usr/sbin/munin19:15:230
176532190,4sleep339-21rcuc/323:10:163
767999188,3cyclictest0-21swapper/321:03:373
767999188,2cyclictest9-21rcu_preempt19:30:223
767999187,3cyclictest16469-21nfs4_client00:20:213
767999184,12cyclictest41-21ksoftirqd/300:30:233
767999184,12cyclictest39-21rcuc/319:10:203
767999183,12cyclictest30187-21/usr/sbin/munin23:40:113
767999181,5cyclictest18245-1kworker/u9:200:19:083
767999181,12cyclictest0-21swapper/323:45:233
767999181,10cyclictest24717-21if_enp2s019:45:313
767999181,10cyclictest24717-21if_enp2s019:45:313
767999181,10cyclictest0-21swapper/322:38:513
767999181,10cyclictest0-21swapper/320:35:193
767399186,3cyclictest0-21swapper/221:49:492
767399182,13cyclictest14420-21grep19:20:272
767399181,6cyclictest0-21swapper/222:12:082
767399181,5cyclictest31625-21ls23:40:232
767399181,5cyclictest29018-21taskset22:20:552
767399181,4cyclictest0-21swapper/219:50:112
767399181,15cyclictest0-21swapper/221:52:022
767399181,15cyclictest0-21swapper/221:15:122
766399185,4cyclictest87350irq/126-enp1s0-22:05:140
766399185,4cyclictest87350irq/126-enp1s0-22:05:140
766399182,15cyclictest0-21swapper/020:19:070
766399181,9cyclictest0-21swapper/000:38:020
7663991814,3cyclictest7-21ksoftirqd/020:30:160
766399181,15cyclictest27162-21awk22:20:010
319342181,2sleep00-21swapper/023:42:160
296692181,2sleep00-21swapper/019:55:260
26242180,3sleep00-21swapper/021:20:500
252252181,2sleep00-21swapper/023:25:510
767999173,9cyclictest41-21ksoftirqd/320:09:053
767999173,12cyclictest41-21ksoftirqd/319:40:213
767999171,9cyclictest0-21swapper/322:03:413
767999171,8cyclictest808-21snmpd20:15:023
767999170,9cyclictest9553-21grep20:25:143
767999170,9cyclictest0-21swapper/321:53:323
767399178,2cyclictest87550irq/128-enp1s0-23:50:182
767399175,3cyclictest10952-21cstates19:15:142
767399175,2cyclictest87550irq/128-enp1s0-22:30:272
767399173,3cyclictest87350irq/126-enp1s0-23:00:222
767399172,9cyclictest0-21swapper/200:10:562
767399171,6cyclictest0-21swapper/223:20:052
767399171,5cyclictest0-21swapper/221:23:162
767399171,4cyclictest0-21swapper/222:25:032
7673991712,3cyclictest26759-21/usr/sbin/munin21:05:242
7673991712,3cyclictest18581-21/usr/sbin/munin22:00:272
7673991712,3cyclictest14908-21timerandwakeup20:35:252
7673991712,2cyclictest21748-21/usr/sbin/munin19:40:142
7673991711,4cyclictest26106-21grep21:00:242
767399171,14cyclictest22282sleep220:08:392
7673991711,3cyclictest29048-21basename19:55:162
7673991710,4cyclictest2896-21sort22:35:232
7673991710,4cyclictest28460-21grep23:35:162
766399178,6cyclictest7-21ksoftirqd/023:15:190
766399178,6cyclictest7-21ksoftirqd/023:15:190
766399176,4cyclictest0-21swapper/022:42:060
766399176,3cyclictest25795-21/usr/sbin/munin23:30:250
766399173,8cyclictest87550irq/128-enp1s0-22:30:250
766399173,2cyclictest87350irq/126-enp1s0-00:30:030
766399171,9cyclictest0-21swapper/019:20:160
766399171,7cyclictest30530-21/usr/sbin/munin20:00:180
766399171,7cyclictest11989-21/usr/sbin/munin21:45:250
766399171,6cyclictest0-21swapper/023:48:480
766399171,5cyclictest808-21snmpd19:44:550
7663991713,3cyclictest0-21swapper/023:05:040
7663991712,4cyclictest0-21swapper/022:59:540
766399171,14cyclictest0-21swapper/019:10:030
7663991711,3cyclictest27571-21cat22:20:110
7663991710,5cyclictest7-21ksoftirqd/022:30:010
278882171,2sleep00-21swapper/019:52:210
132992171,2sleep20-21swapper/220:34:412
97182160,2sleep00-21swapper/020:25:160
767999161,3cyclictest9-21rcu_preempt20:00:213
767999161,12cyclictest29093-21if_enp2s019:55:183
767999161,12cyclictest0-21swapper/320:20:023
767399166,3cyclictest87350irq/126-enp1s0-23:05:152
767399161,9cyclictest0-21swapper/222:16:252
767399161,6cyclictest0-21swapper/220:25:202
767399161,6cyclictest0-21swapper/200:25:162
767399161,6cyclictest0-21swapper/200:25:152
767399161,5cyclictest0-21swapper/200:30:162
7673991612,3cyclictest0-21swapper/219:49:532
7673991612,3cyclictest0-21swapper/219:49:532
7673991612,3cyclictest0-21swapper/219:13:562
767399161,14cyclictest0-21swapper/220:22:422
767399161,14cyclictest0-21swapper/219:26:482
7673991611,3cyclictest23597-21sendmail_mailst20:55:242
7673991611,3cyclictest23597-21sendmail_mailst20:55:242
7673991611,3cyclictest20926-21cut22:05:152
7673991611,3cyclictest20926-21cut22:05:152
7673991610,3cyclictest8212-21/usr/sbin/munin22:50:192
767399160,14cyclictest0-21swapper/222:56:132
767399160,14cyclictest0-21swapper/220:16:202
766399163,11cyclictest28959-21/usr/sbin/munin21:10:130
766399161,9cyclictest0-21swapper/021:58:350
766399161,9cyclictest0-21swapper/020:45:260
766399161,6cyclictest2574-21diskstats23:50:140
766399161,3cyclictest0-21swapper/019:47:370
766399161,3cyclictest0-21swapper/019:47:370
766399161,14cyclictest0-21swapper/000:06:440
766399161,12cyclictest17349-21/usr/sbin/munin19:30:110
327262160,2sleep00-21swapper/020:05:050
114202161,2sleep20-21swapper/221:40:502
767999158,1cyclictest0-21swapper/320:45:143
767999154,3cyclictest87650irq/129-enp1s0-19:39:543
767999151,5cyclictest26957-21expr21:05:123
767399151,5cyclictest808-21snmpd19:39:552
767399151,5cyclictest0-21swapper/220:45:152
767399151,5cyclictest0-21swapper/220:10:132
7673991510,3cyclictest26083-21cat23:30:132
766399151,5cyclictest22357-21/usr/sbin/munin20:55:160
766399151,5cyclictest22357-21/usr/sbin/munin20:55:160
7663991510,3cyclictest22986-21/usr/sbin/munin22:10:150
5482150,2sleep00-21swapper/021:18:410
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional