You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 14:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Sun Apr 26, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
46262236161,24sleep00-21swapper/007:05:060
64952205169,24sleep20-21swapper/207:08:492
63922202167,23sleep30-21swapper/307:07:293
62962199165,23sleep10-21swapper/107:06:161
82921780,5sleep0667799cyclictest09:20:180
1766321600,5sleep0667799cyclictest10:00:120
182742600,4sleep230-21rcuc/208:45:252
80312460,1sleep00-21swapper/012:05:130
121122440,5sleep212115-21latency_hist12:15:012
6688994026,8cyclictest0-21swapper/207:12:522
669399341,24cyclictest0-21swapper/308:16:333
669399341,11cyclictest13430-21irqstats08:35:173
668899341,26cyclictest0-21swapper/212:28:282
668899341,20cyclictest0-21swapper/208:36:442
668899341,17cyclictest0-21swapper/209:22:162
668899341,15cyclictest0-21swapper/210:42:082
668399341,12cyclictest0-21swapper/108:04:461
667799347,13cyclictest30612-21perf10:30:010
667799345,13cyclictest23041-21munin-run12:40:000
668899331,23cyclictest0-21swapper/207:33:072
668899331,19cyclictest0-21swapper/207:23:432
668899331,18cyclictest0-21swapper/209:55:312
668899331,15cyclictest0-21swapper/211:47:472
668899331,15cyclictest0-21swapper/211:47:472
668899331,14cyclictest0-21swapper/212:22:402
668899331,13cyclictest0-21swapper/208:33:362
668399331,24cyclictest0-21swapper/108:26:181
668399331,23cyclictest0-21swapper/109:03:571
668899321,22cyclictest0-21swapper/208:25:402
668899321,21cyclictest0-21swapper/208:20:202
668899321,21cyclictest0-21swapper/207:56:072
668899321,19cyclictest0-21swapper/210:06:232
668899321,19cyclictest0-21swapper/208:16:122
668899321,16cyclictest0-21swapper/211:13:442
668899321,15cyclictest0-21swapper/212:02:112
668899321,15cyclictest0-21swapper/211:43:312
668899321,15cyclictest0-21swapper/208:58:162
668899321,13cyclictest0-21swapper/211:51:032
668899321,13cyclictest0-21swapper/211:05:202
668899320,20cyclictest0-21swapper/209:11:002
668399321,23cyclictest0-21swapper/111:15:411
668399321,18cyclictest0-21swapper/109:17:251
668399320,11cyclictest0-21swapper/107:32:411
667799324,13cyclictest5455-21perf12:00:020
6688993119,3cyclictest25518-21diskstats07:50:152
668899311,24cyclictest0-21swapper/209:31:512
668899311,23cyclictest0-21swapper/210:53:002
668899311,21cyclictest0-21swapper/209:07:482
668899311,20cyclictest0-21swapper/211:25:002
668899311,15cyclictest0-21swapper/211:02:482
668899310,18cyclictest0-21swapper/208:09:512
668899301,20cyclictest0-21swapper/208:54:042
668899301,15cyclictest0-21swapper/212:36:082
668899301,15cyclictest0-21swapper/212:07:472
668899301,15cyclictest0-21swapper/210:01:072
668899301,14cyclictest0-21swapper/209:44:232
668899301,13cyclictest0-21swapper/210:13:432
668899301,13cyclictest0-21swapper/209:35:152
668899301,13cyclictest0-21swapper/209:28:272
668899301,13cyclictest0-21swapper/209:28:272
668899300,11cyclictest0-21swapper/207:47:272
668399301,10cyclictest6916-21expr09:35:131
669399291,25cyclictest0-21swapper/308:47:053
669399291,11cyclictest0-21swapper/310:10:093
6688992916,7cyclictest0-21swapper/212:34:002
668899291,19cyclictest0-21swapper/207:40:032
668899291,14cyclictest0-21swapper/210:34:552
668899291,13cyclictest0-21swapper/211:30:162
668899291,11cyclictest23936-21df09:00:122
668399291,13cyclictest0-21swapper/110:18:221
668399291,12cyclictest1617-21df_inode11:50:131
668399291,12cyclictest0-21swapper/108:33:021
668399291,11cyclictest0-21swapper/110:50:141
667799291,13cyclictest13013-21munin-run09:50:020
667799291,13cyclictest11737-21munin-run07:20:010
667799291,13cyclictest11444-21basename11:00:010
669399280,11cyclictest0-21swapper/310:31:453
6688992817,3cyclictest0-21swapper/211:57:232
668899281,14cyclictest0-21swapper/209:53:032
668899281,13cyclictest0-21swapper/210:26:072
668899281,13cyclictest0-21swapper/208:42:362
668899281,13cyclictest0-21swapper/208:11:202
668899281,13cyclictest0-21swapper/208:11:192
668899281,12cyclictest0-21swapper/210:19:032
668899281,11cyclictest0-21swapper/208:00:252
668899281,11cyclictest0-21swapper/207:18:122
668899280,26cyclictest0-21swapper/210:46:402
668899280,19cyclictest0-21swapper/212:19:192
668899280,17cyclictest0-21swapper/209:16:402
668899280,12cyclictest0-21swapper/207:26:272
668899280,11cyclictest0-21swapper/211:36:082
668399281,9cyclictest0-21swapper/110:30:261
668399281,11cyclictest0-21swapper/109:57:211
668399281,11cyclictest0-21swapper/108:55:221
668399280,21cyclictest0-21swapper/111:21:331
667799287,17cyclictest22727-21munin-run07:45:020
669399271,9cyclictest0-21swapper/309:11:343
669399270,11cyclictest0-21swapper/309:50:423
669399270,11cyclictest0-21swapper/308:31:493
6688992716,3cyclictest0-21swapper/211:28:122
668899271,13cyclictest0-21swapper/210:20:432
668399278,11cyclictest0-21swapper/112:38:221
668399271,9cyclictest0-21swapper/112:20:261
668399271,9cyclictest0-21swapper/108:37:461
6683992710,10cyclictest10169-21kworker/u8:107:20:211
668399270,9cyclictest0-21swapper/110:03:451
668399270,5cyclictest0-21swapper/108:43:261
668399270,11cyclictest0-21swapper/112:00:221
668399270,11cyclictest0-21swapper/110:20:141
284952270,4sleep339-21rcuc/311:35:263
669399261,9cyclictest0-21swapper/312:17:253
669399260,9cyclictest0-21swapper/310:06:503
669399260,11cyclictest0-21swapper/307:38:053
668899261,18cyclictest0-21swapper/207:37:152
668899261,13cyclictest0-21swapper/210:56:562
668899261,11cyclictest0-21swapper/209:45:472
668399265,8cyclictest0-21swapper/111:47:171
668399265,8cyclictest0-21swapper/111:47:171
668399261,9cyclictest0-21swapper/110:57:141
668399261,9cyclictest0-21swapper/110:26:541
668399261,9cyclictest0-21swapper/109:50:051
668399261,9cyclictest0-21swapper/109:10:291
668399261,9cyclictest0-21swapper/107:45:341
668399261,8cyclictest0-21swapper/109:49:251
668399261,12cyclictest0-21swapper/107:41:261
668399261,11cyclictest0-21swapper/110:45:041
668399261,11cyclictest0-21swapper/110:05:411
668399261,11cyclictest0-21swapper/107:10:221
668399261,10cyclictest0-21swapper/112:05:501
668399261,10cyclictest0-21swapper/111:14:571
668399260,9cyclictest0-21swapper/109:07:491
668399260,17cyclictest0-21swapper/111:56:571
668399260,16cyclictest15084-21kworker/u8:110:10:291
668399260,11cyclictest0-21swapper/109:41:091
668399260,11cyclictest0-21swapper/108:18:541
668399260,10cyclictest0-21swapper/111:32:571
668399260,10cyclictest0-21swapper/107:55:141
667799267,16cyclictest31220-21munin-run11:45:010
669399251,9cyclictest0-21swapper/308:07:513
669399251,8cyclictest0-21swapper/311:01:063
669399251,10cyclictest0-21swapper/309:56:223
669399250,8cyclictest0-21swapper/311:32:423
668899258,9cyclictest0-21swapper/210:36:162
668899258,9cyclictest0-21swapper/210:36:162
668399257,11cyclictest87550irq/128-enp1s0-12:30:171
668399256,11cyclictest87350irq/126-enp1s0-08:22:341
668399251,9cyclictest0-21swapper/112:16:181
668399251,9cyclictest0-21swapper/108:51:501
668399251,9cyclictest0-21swapper/108:12:021
668399251,9cyclictest0-21swapper/108:12:021
668399251,10cyclictest0-21swapper/111:08:551
668399250,9cyclictest0-21swapper/112:10:461
668399250,9cyclictest0-21swapper/111:26:571
668399250,11cyclictest0-21swapper/112:27:141
668399250,11cyclictest0-21swapper/111:35:051
667799251,11cyclictest17404-21munin-run10:00:020
669399241,12cyclictest0-21swapper/310:40:053
669399240,10cyclictest0-21swapper/311:50:223
668899240,12cyclictest0-21swapper/211:15:202
668399243,13cyclictest87350irq/126-enp1s0-07:26:331
668399243,12cyclictest87350irq/126-enp1s0-07:50:261
668399241,11cyclictest0-21swapper/108:48:121
668399241,10cyclictest0-21swapper/109:32:531
668399241,10cyclictest0-21swapper/107:19:381
668399240,9cyclictest0-21swapper/107:37:101
668399240,10cyclictest0-21swapper/108:06:581
669399238,7cyclictest0-21swapper/308:58:243
669399238,4cyclictest8408-21awk10:50:233
669399238,3cyclictest155182idleruntime08:40:163
669399237,4cyclictest6021-21diskstats12:00:153
669399231,8cyclictest0-21swapper/310:37:373
669399231,8cyclictest0-21swapper/310:37:373
669399231,16cyclictest0-21swapper/308:13:453
669399231,16cyclictest0-21swapper/308:13:453
669399231,11cyclictest0-21swapper/308:53:393
669399231,11cyclictest0-21swapper/307:46:373
669399230,11cyclictest0-21swapper/311:56:493
669399230,10cyclictest0-21swapper/310:46:093
668399231,9cyclictest0-21swapper/111:03:101
668399230,8cyclictest0-21swapper/110:42:221
668399230,10cyclictest0-21swapper/111:40:411
668399230,10cyclictest0-21swapper/110:36:541
668399230,10cyclictest0-21swapper/110:36:541
669399228,3cyclictest18296-21hddtemp_smartct10:00:163
669399221,8cyclictest0-21swapper/311:09:263
669399221,12cyclictest0-21swapper/307:13:413
669399221,11cyclictest9735-21taskset12:08:373
669399220,8cyclictest0-21swapper/309:31:503
669399220,10cyclictest0-21swapper/311:44:543
668399228,7cyclictest0-21swapper/109:27:051
668399228,7cyclictest0-21swapper/109:27:051
668399221,9cyclictest0-21swapper/109:21:391
667799222,17cyclictest808-21snmpd08:52:170
669399216,3cyclictest0-21swapper/307:16:323
669399213,6cyclictest20714-40dnf12:33:363
669399210,9cyclictest0-21swapper/310:23:413
667799219,7cyclictest0-21swapper/007:49:570
667799211,12cyclictest7260-21/usr/sbin/munin10:50:230
669399204,6cyclictest0-21swapper/309:39:543
669399203,2cyclictest87350irq/126-enp1s0-11:26:353
669399201,6cyclictest0-21swapper/309:24:343
6693992014,3cyclictest10523-21ntp_states10:55:213
669399201,13cyclictest0-21swapper/309:03:143
669399201,11cyclictest0-21swapper/311:46:223
669399201,11cyclictest0-21swapper/311:46:223
669399201,11cyclictest0-21swapper/311:11:383
669399201,10cyclictest0-21swapper/312:13:063
669399201,10cyclictest0-21swapper/307:21:423
669399200,10cyclictest0-21swapper/309:25:363
669399200,10cyclictest0-21swapper/309:25:363
667799206,3cyclictest3308-21ntp_states09:25:220
667799206,3cyclictest3308-21ntp_states09:25:220
667799206,3cyclictest166512expr07:30:120
667799203,9cyclictest0-21swapper/012:23:320
667799202,10cyclictest22649-21/usr/sbin/munin11:25:240
6677992014,3cyclictest21088-21gltestperf11:20:160
667799201,16cyclictest81ktimersoftd/008:30:180
667799201,16cyclictest17816-21tr12:25:220
667799201,11cyclictest13859-21/usr/sbin/munin11:05:190
669399197,4cyclictest0-21swapper/309:47:063
669399197,2cyclictest27039-21sendmail_mailqu09:05:243
669399196,3cyclictest0-21swapper/312:39:133
669399196,3cyclictest0-21swapper/312:24:313
669399196,3cyclictest0-21swapper/309:40:043
669399196,3cyclictest0-21swapper/308:24:313
669399196,3cyclictest0-21swapper/307:59:273
669399191,8cyclictest27784-21diskmemload10:29:183
669399191,6cyclictest0-21swapper/311:22:223
669399191,12cyclictest0-21swapper/312:25:183
669399191,11cyclictest0-21swapper/309:18:463
669399190,12cyclictest0-21swapper/307:43:453
667799199,2cyclictest18254-21/usr/sbin/munin11:15:140
667799193,10cyclictest27784-21diskmemload12:33:350
667799191,6cyclictest0-21swapper/009:39:450
6677991912,4cyclictest4905-21grep09:30:140
667799191,13cyclictest0-21swapper/007:56:200
667799191,12cyclictest0-21swapper/012:00:190
12662190,1sleep00-21swapper/011:50:130
669399185,3cyclictest0-21swapper/308:27:233
669399182,3cyclictest0-21swapper/311:19:513
669399181,11cyclictest0-21swapper/310:19:123
669399181,11cyclictest0-21swapper/307:32:503
669399181,10cyclictest0-21swapper/308:00:243
667799187,2cyclictest28389-21kworker/u8:210:30:250
667799186,4cyclictest22305-1kworker/u9:212:16:040
667799186,3cyclictest87550irq/128-enp1s0-10:15:160
667799182,15cyclictest0-21swapper/010:14:290
667799181,6cyclictest0-21swapper/011:12:200
667799181,11cyclictest0-21swapper/010:20:150
6677991810,5cyclictest0-21swapper/009:10:240
296532181,12sleep029917-21df08:00:130
669399175,3cyclictest0-21swapper/307:52:553
667799176,3cyclictest28736-21sshd11:36:330
667799176,3cyclictest27114-21sshd07:54:350
667799175,3cyclictest13221-21tail08:35:140
667799173,3cyclictest4469-21sshd10:40:330
667799171,9cyclictest808-21snmpd09:07:360
667799171,7cyclictest0-21swapper/011:31:310
667799171,7cyclictest0-21swapper/010:49:460
667799171,6cyclictest0-21swapper/008:45:160
667799171,6cyclictest0-21swapper/008:25:170
667799171,5cyclictest0-21swapper/012:13:350
667799171,4cyclictest21807-21munin-run10:10:020
667799171,3cyclictest0-21swapper/007:24:500
6677991713,3cyclictest0-21swapper/010:39:200
6677991713,3cyclictest0-21swapper/010:39:190
6677991712,3cyclictest1969-21sh08:10:140
6677991712,3cyclictest1969-21sh08:10:140
667799171,15cyclictest0-21swapper/007:12:080
6677991711,4cyclictest12268-21grep11:00:150
6677991711,3cyclictest3823-21/usr/sbin/munin08:15:200
667799171,13cyclictest31442-21/usr/sbin/munin11:45:260
667799171,13cyclictest31442-21/usr/sbin/munin11:45:260
6677991710,4cyclictest16789-21munin-run08:45:010
669399161,9cyclictest0-21swapper/307:25:063
667799169,4cyclictest32385-21grep08:05:170
667799161,6cyclictest0-21swapper/009:52:130
667799161,6cyclictest0-21swapper/007:35:180
667799161,5cyclictest0-21swapper/009:40:160
6677991612,3cyclictest0-21swapper/009:02:180
6677991612,2cyclictest31278-21ntp_states09:15:230
667799161,14cyclictest0-21swapper/008:59:030
6677991610,3cyclictest6653-21head08:20:160
6677991610,3cyclictest14163-21/usr/sbin/munin07:25:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional