You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 04:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Aug 27, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151742243166,25sleep20-21swapper/219:06:342
151742243166,25sleep20-21swapper/219:06:342
152602231162,56sleep00-21swapper/019:07:410
152602231162,56sleep00-21swapper/019:07:410
151002227156,22sleep10-21swapper/119:05:361
151002227156,22sleep10-21swapper/119:05:361
153312201166,23sleep30-21swapper/319:08:363
153312201166,23sleep30-21swapper/319:08:363
2057721270,4sleep11553099cyclictest20:30:241
15535999385,3cyclictest0-21swapper/200:37:182
240282520,3sleep224031-21idleruntime-cro21:55:012
15535995046,3cyclictest0-21swapper/220:33:582
284412480,3sleep3401ktimersoftd/322:05:023
235662470,4sleep341-21ksoftirqd/320:40:023
210212460,1sleep20-21swapper/219:20:102
15541994027,3cyclictest0-21swapper/319:33:033
1554199401,31cyclictest0-21swapper/323:58:593
15541993824,8cyclictest0-21swapper/300:23:273
15541993623,8cyclictest0-21swapper/319:48:273
1554199361,19cyclictest4237-21diskmemload23:28:273
15541993526,7cyclictest0-21swapper/319:56:353
15541993520,3cyclictest0-21swapper/320:27:033
1554199351,27cyclictest0-21swapper/322:17:593
1554199351,15cyclictest0-21swapper/321:45:233
1553599351,27cyclictest0-21swapper/221:44:132
1552499359,12cyclictest8647-21idleruntime-cro21:20:010
1552499354,15cyclictest22452-21munin-run23:05:010
15524993510,12cyclictest25601-21munin-run20:45:010
15541993426,3cyclictest0-21swapper/320:07:393
1553599344,3cyclictest87650irq/129-enp1s0-23:05:182
1554199331,14cyclictest0-21swapper/323:41:153
1553599332,7cyclictest24088-40dnf19:25:252
1552499338,11cyclictest14619-21awk20:20:010
1552499337,13cyclictest18058-21awk22:55:010
86342320,2sleep18637-21latency_hist21:20:011
1554199321,23cyclictest0-21swapper/322:49:513
1554199321,19cyclictest0-21swapper/320:23:513
1554199321,18cyclictest4237-21diskmemload00:26:033
1554199321,14cyclictest0-21swapper/323:37:313
1554199321,14cyclictest0-21swapper/322:10:513
1553599321,14cyclictest0-21swapper/222:48:132
15541993118,3cyclictest0-21swapper/322:37:513
15541993118,3cyclictest0-21swapper/322:37:513
1553599311,19cyclictest0-21swapper/220:20:252
1553599311,14cyclictest0-21swapper/219:49:012
1553599311,13cyclictest0-21swapper/220:11:172
1553599310,24cyclictest0-21swapper/223:00:172
1553599310,13cyclictest0-21swapper/223:54:412
15541993018,3cyclictest0-21swapper/323:54:353
15541993018,3cyclictest0-21swapper/319:52:553
15541993018,2cyclictest659-21kworker/u8:219:26:353
15541993017,3cyclictest0-21swapper/321:10:473
15541993015,8cyclictest0-21swapper/300:07:393
1554199301,17cyclictest4237-21diskmemload21:58:153
1554199301,13cyclictest23765-21cpuspeed_turbos20:40:113
1554199301,13cyclictest0-21swapper/322:07:033
1554199301,12cyclictest0-21swapper/323:19:193
1553599301,13cyclictest26568-21ntp_states00:25:212
1553599301,12cyclictest0-21swapper/219:40:292
15535993010,2cyclictest87650irq/129-enp1s0-23:55:192
15541992918,3cyclictest0-21swapper/322:51:273
15541992917,3cyclictest0-21swapper/322:43:033
1554199291,18cyclictest0-21swapper/321:34:433
1554199291,13cyclictest0-21swapper/321:50:273
1554199291,12cyclictest0-21swapper/323:31:153
1553599291,20cyclictest0-21swapper/219:10:432
1553599291,13cyclictest0-21swapper/223:29:132
1553599291,13cyclictest0-21swapper/221:45:412
1553599290,17cyclictest0-21swapper/219:17:012
15541992812,6cyclictest0-21swapper/320:52:313
1554199281,13cyclictest0-21swapper/323:02:353
1554199281,13cyclictest0-21swapper/321:29:313
1554199281,12cyclictest0-21swapper/321:42:113
1554199281,12cyclictest0-21swapper/300:01:353
1554199281,11cyclictest0-21swapper/322:56:473
15535992812,3cyclictest11517-21date22:40:012
15535992812,3cyclictest11517-21date22:40:012
1553599281,14cyclictest0-21swapper/220:58:332
1553599281,14cyclictest0-21swapper/220:02:452
15535992811,3cyclictest1978-21df19:50:132
1553599281,13cyclictest0-21swapper/222:13:052
1553599281,13cyclictest0-21swapper/220:51:132
1553599281,13cyclictest0-21swapper/200:33:012
1553599281,12cyclictest10200-21grep21:20:252
1553599281,12cyclictest0-21swapper/221:57:412
1553599281,11cyclictest0-21swapper/222:07:392
140412280,4sleep29-21rcu_preempt21:30:212
15541992722,3cyclictest0-21swapper/319:20:433
15541992716,3cyclictest0-21swapper/321:24:193
15541992716,3cyclictest0-21swapper/320:16:353
15541992714,8cyclictest0-21swapper/320:49:113
15541992714,3cyclictest0-21swapper/320:55:313
1554199271,18cyclictest0-21swapper/320:12:353
1554199271,12cyclictest0-21swapper/323:05:473
1553599271,12cyclictest7472-21awk21:15:172
1553599271,12cyclictest0-21swapper/220:49:012
1553599271,11cyclictest0-21swapper/221:02:012
1553599271,11cyclictest0-21swapper/219:33:052
1554199268,2cyclictest0-21swapper/321:37:273
1554199261,11cyclictest0-21swapper/319:41:113
1554199261,10cyclictest0-21swapper/322:26:513
15541992610,4cyclictest808-21snmpd19:37:153
1553599269,3cyclictest0-21swapper/219:36:412
1553599261,12cyclictest8958-21ntp_states23:45:212
1553599261,12cyclictest8958-21ntp_states23:45:212
1553599261,12cyclictest0-21swapper/223:24:012
1553599261,12cyclictest0-21swapper/223:11:332
1553599261,12cyclictest0-21swapper/220:37:172
1553599261,11cyclictest0-21swapper/221:25:052
1553599261,11cyclictest0-21swapper/221:08:452
1553599260,12cyclictest0-21swapper/222:57:212
1552499267,16cyclictest22789-21munin-run19:25:010
1552499261,13cyclictest20256-21idleruntime-cro23:00:010
1554199259,3cyclictest0-21swapper/320:33:553
15541992517,3cyclictest0-21swapper/319:19:073
15541992516,3cyclictest0-21swapper/320:00:273
15541992516,3cyclictest0-21swapper/300:12:233
15541992515,3cyclictest0-21swapper/323:20:433
15541992511,3cyclictest4237-21diskmemload00:31:033
1554199251,12cyclictest0-21swapper/323:14:273
1554199251,12cyclictest0-21swapper/322:23:233
1554199251,12cyclictest0-21swapper/300:39:233
1554199251,12cyclictest0-21swapper/300:15:073
1554199251,11cyclictest0-21swapper/323:45:353
1554199251,11cyclictest0-21swapper/323:45:353
1554199251,11cyclictest0-21swapper/321:15:433
1553599259,3cyclictest0-21swapper/200:20:292
1553599259,3cyclictest0-21swapper/200:17:252
1553599251,9cyclictest3842-21/usr/sbin/munin19:55:252
1553599251,15cyclictest7280-21users23:40:252
15535992510,3cyclictest0-21swapper/220:26:372
1553599250,10cyclictest0-21swapper/223:35:172
1552499253,10cyclictest0-21swapper/022:30:010
1554199249,3cyclictest0-21swapper/321:06:553
1554199248,3cyclictest0-21swapper/321:04:433
15541992416,3cyclictest0-21swapper/319:12:273
1554199241,12cyclictest0-21swapper/322:34:033
1553599249,3cyclictest0-21swapper/220:08:212
1553599249,3cyclictest0-21swapper/200:07:212
1553599248,3cyclictest0-21swapper/222:25:492
1553599248,3cyclictest0-21swapper/222:17:412
1553599248,2cyclictest0-21swapper/200:10:132
1553599241,16cyclictest0-21swapper/222:54:372
15535992410,3cyclictest0-21swapper/220:16:132
15535992410,3cyclictest0-21swapper/200:01:492
1553099241,16cyclictest0-21swapper/122:23:111
15530992411,5cyclictest31069-21proc_pri00:35:221
15530992411,11cyclictest24530-21kworker/u8:023:51:041
1552499241,12cyclictest21836-21awk21:50:010
1553599239,2cyclictest0-21swapper/222:20:212
1553599232,13cyclictest0-21swapper/221:13:472
15535992312,3cyclictest87650irq/129-enp1s0-22:40:132
1553599230,12cyclictest0-21swapper/221:36:452
1553099238,7cyclictest0-21swapper/121:02:351
15530992317,4cyclictest28428-21perf22:05:011
15530992317,3cyclictest11354-21/usr/sbin/munin20:10:191
15530992313,2cyclictest29716-21ntp_states22:05:211
1553099231,16cyclictest0-21swapper/122:49:571
1553599228,3cyclictest0-21swapper/222:31:492
1553599227,3cyclictest0-21swapper/223:15:052
1553099226,3cyclictest0-21swapper/100:26:011
15530992212,3cyclictest19819-21awk22:55:251
1553099221,18cyclictest16745-21hddtemp_smartct22:50:171
1553099221,15cyclictest0-21swapper/100:13:261
1553099220,16cyclictest87650irq/129-enp1s0-21:22:371
1552499224,6cyclictest11-21rcuc/022:35:590
1552499224,6cyclictest11-21rcuc/022:35:590
1553599219,3cyclictest0-21swapper/222:00:252
1553599211,13cyclictest0-21swapper/223:30:452
1553099216,3cyclictest19420-21sshd21:42:301
1553099211,18cyclictest0-21swapper/120:08:041
1553099211,17cyclictest5470-21munin-run23:40:011
1553099211,14cyclictest0-21swapper/122:18:151
1553099211,14cyclictest0-21swapper/121:59:351
1553099211,14cyclictest0-21swapper/100:01:541
1553099211,11cyclictest27985-21memory23:15:221
1553099211,11cyclictest27548-21tr19:35:121
1553099211,11cyclictest17054-21cpu20:25:121
1553099211,10cyclictest32200-21perf21:00:011
1553099210,15cyclictest87650irq/129-enp1s0-22:26:431
1552499218,10cyclictest13108-21uname23:55:180
1552499212,6cyclictest11-21rcuc/020:50:240
1552499211,4cyclictest0-21swapper/022:46:460
15524992112,2cyclictest4242-21munin-run21:10:010
1552499211,13cyclictest87350irq/126-enp1s0-22:20:110
1552499211,13cyclictest11-21rcuc/022:15:150
1552499211,12cyclictest27066-21/usr/sbin/munin23:15:230
147262210,1sleep00-21swapper/000:00:130
238922200,5sleep011-21rcuc/023:05:220
1553099209,3cyclictest7058-21netstat20:00:181
1553099207,7cyclictest0-21swapper/123:45:201
1553099207,7cyclictest0-21swapper/123:45:201
1553099207,11cyclictest30351-21kworker/u8:222:10:061
15530992012,3cyclictest0-21swapper/123:58:241
1553099201,14cyclictest0-21swapper/123:00:591
1553099201,14cyclictest0-21swapper/122:42:321
1553099201,14cyclictest0-21swapper/121:54:201
1553099201,14cyclictest0-21swapper/100:20:441
1553099201,14cyclictest0-21swapper/100:06:081
15530992011,2cyclictest25733-21grep19:30:161
1553099201,11cyclictest27747-21cpu00:30:091
1553099201,11cyclictest22659-21grep20:35:231
1553099200,4cyclictest0-21swapper/122:38:471
1553099200,4cyclictest0-21swapper/122:38:471
1552499209,3cyclictest11-21rcuc/023:40:140
1552499202,10cyclictest25505-21/usr/sbin/munin00:25:120
1552499202,10cyclictest24247-21/usr/sbin/munin21:55:210
1552499201,11cyclictest7890-21/usr/sbin/munin23:45:120
1552499201,11cyclictest7890-21/usr/sbin/munin23:45:120
1552499201,11cyclictest22051-21/usr/sbin/munin21:50:260
1552499201,11cyclictest21111-21/usr/sbin/munin00:15:130
316272192,2sleep00-21swapper/000:37:060
277562190,1sleep00-21swapper/019:35:140
1553099197,3cyclictest0-21swapper/123:06:491
1553099196,3cyclictest17060-21systemd-journal19:10:231
15530991916,2cyclictest0-21swapper/119:54:421
15530991915,3cyclictest0-21swapper/123:41:251
1553099191,2cyclictest0-21swapper/119:42:461
1553099191,17cyclictest0-21swapper/121:34:291
1553099191,17cyclictest0-21swapper/121:26:521
1553099191,17cyclictest0-21swapper/120:16:411
1553099191,16cyclictest0-21swapper/123:10:131
1553099191,16cyclictest0-21swapper/121:45:141
1553099191,10cyclictest27437-21chrt20:45:331
1553099191,10cyclictest23693-21smartctl19:25:171
1553099191,10cyclictest0-21swapper/119:20:171
1553099190,17cyclictest0-21swapper/120:23:131
1553099190,14cyclictest0-21swapper/100:16:501
1552499199,6cyclictest0-21swapper/022:04:560
1552499199,3cyclictest87550irq/128-enp1s0-00:14:300
1552499196,3cyclictest31977-21kworker/0:221:04:410
1552499195,3cyclictest0-21swapper/021:22:000
1552499194,14cyclictest0-21swapper/020:10:010
1552499193,14cyclictest31588-21awk19:45:010
1552499193,10cyclictest0-21swapper/022:34:170
1552499192,8cyclictest11-21rcuc/021:35:220
1552499192,3cyclictest0-21swapper/021:12:140
1552499191,9cyclictest30375-21cut23:20:220
1552499191,6cyclictest0-21swapper/019:45:250
1552499191,3cyclictest9-21rcu_preempt23:25:590
1552499191,10cyclictest4237-21diskmemload00:21:480
15524991910,2cyclictest165050irq/12-ttyS000:05:500
1552499190,3cyclictest11-21rcuc/019:15:010
1553599181,13cyclictest0-21swapper/220:40:252
1553099186,4cyclictest0-21swapper/123:30:131
1553099182,13cyclictest32718-21smart_sda23:25:231
1553099181,9cyclictest30931-21sshd23:21:161
1553099181,9cyclictest20253-21runrttasks19:15:491
1553099181,6cyclictest8193-21memory22:30:181
15530991816,1cyclictest0-21swapper/119:45:181
15530991815,2cyclictest0-21swapper/120:50:101
1553099181,16cyclictest0-21swapper/121:13:081
1553099181,16cyclictest0-21swapper/120:41:581
1553099181,16cyclictest0-21swapper/119:56:501
1553099181,15cyclictest0-21swapper/121:35:511
1553099180,2cyclictest0-21swapper/121:05:141
1552499183,6cyclictest0-21swapper/021:33:070
1552499183,3cyclictest11303-21expr21:25:130
1552499182,13cyclictest3914-21grep23:35:140
1552499181,9cyclictest0-21swapper/022:11:250
1552499181,8cyclictest20825-21users20:30:260
1552499181,7cyclictest87350irq/126-enp1s0-00:30:100
1552499181,5cyclictest18719-21ntp_states21:40:210
1552499181,4cyclictest4237-21diskmemload22:05:130
15524991812,3cyclictest27809-21basename20:50:010
15524991811,4cyclictest22041-21grep20:35:160
1552499181,12cyclictest0-21swapper/023:50:390
1552499181,11cyclictest0-21swapper/023:30:200
1552499180,12cyclictest0-21swapper/022:40:220
1552499174,5cyclictest11-21rcuc/020:20:140
1552499174,2cyclictest11-21rcuc/020:25:130
1552499172,6cyclictest87550irq/128-enp1s0-20:04:230
15524991713,3cyclictest0-21swapper/023:11:160
15524991712,3cyclictest4653-21irqstats19:55:180
15524991710,4cyclictest0-21swapper/019:50:140
196462161,12sleep019681-21ntp_states19:15:200
1552499161,6cyclictest0-21swapper/019:30:240
15524991611,3cyclictest17843-21munin-node20:55:130
1552499161,12cyclictest10439-21/usr/sbin/munin20:10:120
15524991510,3cyclictest24132-21pmu-power19:25:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional