You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 03:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Jul 26, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
314552240165,25sleep20-21swapper/219:08:042
313862240162,25sleep00-21swapper/019:07:090
312672236163,24sleep10-21swapper/119:05:381
313372229154,24sleep30-21swapper/319:06:323
31695998784,2cyclictest0-21swapper/220:52:362
31695998784,2cyclictest0-21swapper/200:35:332
31695998278,3cyclictest0-21swapper/221:44:022
31695998178,2cyclictest0-21swapper/223:14:392
324702430,1sleep30-21swapper/322:50:143
23562400,1sleep00-21swapper/022:55:130
3168099389,13cyclictest27004-21awk21:25:010
47362370,4sleep00-21swapper/023:00:140
3170099371,7cyclictest0-21swapper/300:30:213
3170099361,6cyclictest0-21swapper/323:50:093
3170099361,17cyclictest0-21swapper/300:36:473
3170099361,10cyclictest14025-21/usr/sbin/munin20:55:153
3170099351,21cyclictest0-21swapper/321:16:433
3169599351,25cyclictest29966-21cat22:45:012
3170099341,26cyclictest0-21swapper/321:46:113
3170099341,26cyclictest0-21swapper/321:46:103
3170099341,19cyclictest0-21swapper/321:33:433
3168099349,12cyclictest16624-21munin-run22:15:000
3170099331,24cyclictest0-21swapper/321:23:433
3170099331,22cyclictest0-21swapper/321:35:303
3170099331,21cyclictest0-21swapper/323:08:233
3170099331,21cyclictest0-21swapper/322:56:433
3170099331,20cyclictest0-21swapper/323:48:033
3170099331,20cyclictest0-21swapper/323:12:353
3170099331,10cyclictest21243-21/usr/sbin/munin22:25:193
3168099335,13cyclictest21019-21munin-run22:25:010
3170099321,20cyclictest0-21swapper/321:50:313
3170099321,19cyclictest0-21swapper/323:40:113
3170099321,19cyclictest0-21swapper/323:00:433
3169599321,22cyclictest0-21swapper/223:09:212
3169599321,12cyclictest0-21swapper/200:25:272
3169599320,21cyclictest0-21swapper/223:30:112
247472320,4sleep232-21ksoftirqd/220:05:152
3170099311,20cyclictest0-21swapper/323:22:313
3170099311,20cyclictest0-21swapper/323:16:593
3170099311,19cyclictest0-21swapper/300:25:343
3170099311,13cyclictest0-21swapper/319:12:033
3170099311,12cyclictest0-21swapper/319:49:513
3170099310,20cyclictest0-21swapper/300:12:313
3170099310,19cyclictest0-21swapper/300:00:513
3169599311,22cyclictest0-21swapper/222:26:232
3169599311,21cyclictest0-21swapper/222:12:212
3169599311,11cyclictest0-21swapper/223:20:592
3169599311,11cyclictest0-21swapper/200:14:552
3170099301,3cyclictest0-21swapper/322:42:213
3170099301,21cyclictest0-21swapper/300:17:553
3170099301,19cyclictest0-21swapper/320:37:353
3170099301,19cyclictest0-21swapper/320:37:353
3170099301,13cyclictest0-21swapper/323:57:513
3170099301,12cyclictest14770-21timerandwakeup19:40:393
3170099301,12cyclictest0-21swapper/323:32:353
3170099301,12cyclictest0-21swapper/322:21:033
3170099301,11cyclictest0-21swapper/322:49:113
3169599301,15cyclictest0-21swapper/222:08:152
3169599301,12cyclictest0-21swapper/222:50:352
3169599301,12cyclictest0-21swapper/222:47:512
3169599301,11cyclictest0-21swapper/221:20:032
3168099305,12cyclictest9080-21munin-run00:25:010
3170099291,19cyclictest0-21swapper/300:07:593
3170099291,13cyclictest0-21swapper/321:55:313
3170099291,13cyclictest0-21swapper/321:12:233
3170099291,12cyclictest0-21swapper/323:26:033
3170099291,12cyclictest0-21swapper/322:08:553
3170099291,11cyclictest0-21swapper/323:37:473
3170099291,11cyclictest0-21swapper/322:38:033
3170099291,11cyclictest0-21swapper/322:11:593
3170099291,11cyclictest0-21swapper/320:26:553
3169599291,12cyclictest0-21swapper/223:16:172
3169599291,12cyclictest0-21swapper/222:31:232
3169599291,11cyclictest0-21swapper/222:00:392
3169599291,10cyclictest0-21swapper/222:24:492
3169599290,22cyclictest0-21swapper/221:46:092
3169599290,22cyclictest0-21swapper/221:46:092
3169599290,13cyclictest0-21swapper/220:59:512
3169599290,10cyclictest0-21swapper/221:31:312
3170099281,9cyclictest0-21swapper/319:25:113
3170099281,9cyclictest0-21swapper/319:25:113
3170099281,12cyclictest0-21swapper/322:16:353
3170099281,12cyclictest0-21swapper/320:14:433
3170099281,12cyclictest0-21swapper/320:00:033
3170099281,10cyclictest12172-21diskstats20:50:153
3169599281,10cyclictest0-21swapper/221:01:222
3169599281,10cyclictest0-21swapper/219:41:072
3169599281,10cyclictest0-21swapper/200:15:272
3169599280,15cyclictest0-21swapper/223:44:262
3169599280,12cyclictest0-21swapper/221:10:582
3169599280,11cyclictest0-21swapper/221:59:592
3169599280,11cyclictest0-21swapper/200:21:412
304682280,2sleep10-21swapper/121:30:221
3170099271,12cyclictest0-21swapper/321:43:063
3170099271,11cyclictest0-21swapper/300:21:383
3170099271,10cyclictest0-21swapper/320:09:073
3170099271,10cyclictest0-21swapper/319:35:553
3169599271,9cyclictest0-21swapper/221:18:052
3169599271,9cyclictest0-21swapper/219:16:392
3169599271,12cyclictest0-21swapper/221:51:332
3169599271,12cyclictest0-21swapper/200:08:152
3169599271,11cyclictest0-21swapper/220:04:272
3169599271,10cyclictest20397-21diskmemload22:19:332
3169599270,12cyclictest0-21swapper/220:35:472
3169599270,12cyclictest0-21swapper/220:35:472
3169599270,10cyclictest0-21swapper/219:33:152
3168099272,12cyclictest1221-21idleruntime-cro21:40:010
3170099261,12cyclictest0-21swapper/319:52:353
3170099261,11cyclictest0-21swapper/322:33:073
3170099261,10cyclictest0-21swapper/322:00:473
3170099260,10cyclictest0-21swapper/321:29:423
3170099260,10cyclictest0-21swapper/320:32:113
3170099260,10cyclictest0-21swapper/320:23:073
3169599261,12cyclictest0-21swapper/221:05:022
3169599261,10cyclictest0-21swapper/223:03:552
3169599260,17cyclictest0-21swapper/221:37:072
3169599260,11cyclictest0-21swapper/223:57:272
3169599260,11cyclictest0-21swapper/223:50:312
3169599260,11cyclictest0-21swapper/223:25:312
3169599260,11cyclictest0-21swapper/220:20:512
3169599260,10cyclictest0-21swapper/220:11:312
3169599260,10cyclictest0-21swapper/219:29:232
3169599260,10cyclictest0-21swapper/219:29:232
3169099265,16cyclictest3245-40dnf22:55:251
3169099261,10cyclictest0-21swapper/120:53:391
25862260,7sleep011-21rcuc/000:10:010
3170099251,11cyclictest0-21swapper/319:57:033
3169599258,3cyclictest29378-21cat21:30:012
3169599251,11cyclictest0-21swapper/223:36:192
3169599251,11cyclictest0-21swapper/219:56:552
3169599251,11cyclictest0-21swapper/200:02:112
3169599251,10cyclictest0-21swapper/219:38:032
3169599250,8cyclictest0-21swapper/200:32:152
3169599250,11cyclictest0-21swapper/223:45:342
3169599250,11cyclictest0-21swapper/222:59:472
3169599250,11cyclictest0-21swapper/220:17:552
3169599250,11cyclictest0-21swapper/219:46:352
3169599250,10cyclictest0-21swapper/222:38:072
3169099251,4cyclictest21629-21turbostat.cron23:40:011
3169099251,21cyclictest8816-21cut23:10:121
3168099259,7cyclictest755-21rngd23:18:430
3168099251,12cyclictest1499-21idleruntime-cro19:15:010
3170099241,11cyclictest0-21swapper/321:05:073
3169599241,8cyclictest0-21swapper/220:30:072
3169599240,10cyclictest0-21swapper/220:29:232
3169099242,10cyclictest0-21swapper/123:50:171
3169099241,20cyclictest27503-21expr21:25:131
3169099241,11cyclictest1034-21expr20:25:121
3168099245,16cyclictest24806-21awk21:20:020
3168099241,12cyclictest31386-21munin-run21:35:010
3168099241,10cyclictest11475-21kworker/0:321:02:350
3170099231,11cyclictest0-21swapper/320:47:553
3170099230,10cyclictest0-21swapper/320:40:433
3169599237,3cyclictest0-21swapper/219:53:112
3169599231,9cyclictest0-21swapper/219:11:012
3169099239,3cyclictest32579-21df_inode22:50:151
3169099231,20cyclictest0-21swapper/122:30:221
3169099231,19cyclictest5559-21sed19:20:191
3169099231,11cyclictest17783-21expr23:30:141
3169099231,10cyclictest24847-21sed21:20:021
3169099231,10cyclictest0-21swapper/120:20:001
31680992313,3cyclictest21635-21munin-run23:40:010
3168099231,11cyclictest8451-21munin-run23:10:010
3168099231,11cyclictest20397-21diskmemload22:05:170
3170099221,10cyclictest0-21swapper/319:33:273
3169599227,3cyclictest0-21swapper/219:20:092
3169599221,10cyclictest0-21swapper/220:45:332
3169099222,18cyclictest0-21swapper/120:56:561
3169099222,18cyclictest0-21swapper/100:26:501
3169099221,9cyclictest0-21swapper/121:37:341
3169099221,7cyclictest25575-21sshd20:05:231
3169099221,4cyclictest0-21swapper/121:59:361
3169099221,11cyclictest4694-21munin-run00:15:011
3169099221,10cyclictest5869-21egrep21:50:131
3169099220,6cyclictest0-21swapper/122:23:061
3169099220,5cyclictest0-21swapper/122:36:271
3169099220,5cyclictest0-21swapper/122:09:361
3169099220,5cyclictest0-21swapper/100:34:511
3169099220,14cyclictest0-21swapper/100:38:011
3168099221,15cyclictest0-21swapper/023:22:090
3170099211,13cyclictest0-21swapper/321:01:313
3170099211,13cyclictest0-21swapper/320:15:433
3170099211,13cyclictest0-21swapper/319:18:233
3170099210,10cyclictest0-21swapper/319:22:553
3169599210,9cyclictest0-21swapper/220:43:412
3169099217,2cyclictest5003-21latency_hist20:35:001
3169099216,3cyclictest21577-21df22:25:141
3169099211,7cyclictest8093-21taskset23:05:341
31690992116,3cyclictest6710-21grep20:35:271
31690992116,3cyclictest6710-21grep20:35:261
31690992116,3cyclictest0-21swapper/119:15:021
3169099211,18cyclictest0-21swapper/119:58:471
3169099211,18cyclictest0-21swapper/119:38:311
3169099211,17cyclictest21893-21smartctl21:10:241
3169099211,10cyclictest16301-21wc19:45:181
3169099211,10cyclictest14818-21chrt23:22:181
3169099210,4cyclictest0-21swapper/122:13:281
3169099210,4cyclictest0-21swapper/119:52:251
3168099219,3cyclictest0-21swapper/022:38:140
3168099218,7cyclictest0-21swapper/023:25:270
3168099217,2cyclictest0-21swapper/020:15:230
3168099216,8cyclictest0-21swapper/021:58:180
3168099211,8cyclictest1713-21cstates21:40:140
3168099211,6cyclictest0-21swapper/023:13:360
31680992115,4cyclictest28587-21awk20:15:010
3168099211,14cyclictest0-21swapper/021:47:160
3168099211,14cyclictest0-21swapper/021:47:160
3169099207,10cyclictest30440-40updatedb00:00:021
3169099205,3cyclictest87550irq/128-enp1s0-22:00:151
3169099202,16cyclictest0-21swapper/119:27:041
3169099202,16cyclictest0-21swapper/119:27:041
3169099201,8cyclictest0-21swapper/123:40:241
31690992016,3cyclictest0-21swapper/120:10:251
3169099201,4cyclictest0-21swapper/121:45:271
3169099201,4cyclictest0-21swapper/121:45:271
31690992013,4cyclictest16885-21cut21:00:171
3169099201,2cyclictest0-21swapper/123:27:391
3169099201,18cyclictest0-21swapper/123:56:571
3169099200,5cyclictest0-21swapper/121:44:171
3169099200,4cyclictest0-21swapper/100:16:501
3169099200,3cyclictest0-21swapper/121:21:401
3169099200,18cyclictest0-21swapper/120:45:021
3169099200,10cyclictest32186-21latency_hist22:50:011
3168099209,7cyclictest0-21swapper/022:27:300
3168099209,3cyclictest0-21swapper/022:04:190
3168099209,3cyclictest0-21swapper/000:02:520
3168099206,1cyclictest0-21swapper/000:30:190
3168099205,4cyclictest0-21swapper/023:34:260
3168099201,9cyclictest0-21swapper/000:11:480
3168099201,6cyclictest25275-21sshd22:33:250
3168099201,6cyclictest0-21swapper/023:57:140
31680992015,3cyclictest12159-21/usr/sbin/munin20:50:140
31680992014,1cyclictest0-21swapper/023:50:130
3168099200,5cyclictest0-21swapper/021:10:160
3169099192,16cyclictest0-21swapper/119:43:381
31690991915,3cyclictest0-21swapper/122:44:431
31690991915,3cyclictest0-21swapper/119:10:261
31690991915,3cyclictest0-21swapper/100:20:031
3169099191,2cyclictest0-21swapper/120:23:341
31690991912,4cyclictest818-21diskstats00:05:181
3169099191,17cyclictest0-21swapper/120:01:111
3169099190,4cyclictest0-21swapper/123:17:211
3169099190,10cyclictest0-21swapper/123:00:031
3168099199,4cyclictest827-21runrttasks00:18:480
3168099199,1cyclictest0-21swapper/020:40:210
3168099195,7cyclictest0-21swapper/022:47:070
3168099195,3cyclictest0-21swapper/023:47:590
3168099195,2cyclictest0-21swapper/022:15:230
3168099191,6cyclictest6922-21ntp_states21:50:230
3168099191,6cyclictest0-21swapper/022:40:200
3168099191,6cyclictest0-21swapper/021:27:490
3168099191,5cyclictest0-21swapper/019:45:130
3168099191,5cyclictest0-21swapper/000:25:150
3168099191,16cyclictest0-21swapper/020:00:420
3168099191,15cyclictest9809-21cstates20:45:130
3169099181,5cyclictest24055-21/usr/sbin/munin23:45:131
3169099181,4cyclictest17312-21fschecks_count22:15:161
3169099181,2cyclictest0-21swapper/121:06:231
3169099181,16cyclictest0-21swapper/120:46:311
3169099181,16cyclictest0-21swapper/119:31:121
3168099183,5cyclictest2353-21grep20:25:260
3168099183,3cyclictest6758-21unixbench_multi20:35:280
3168099183,3cyclictest6758-21unixbench_multi20:35:280
3168099183,12cyclictest9016-21/usr/sbin/munin19:30:160
3168099182,13cyclictest3040-21/usr/sbin/munin20:30:160
3168099181,6cyclictest0-21swapper/022:52:400
3168099181,6cyclictest0-21swapper/000:37:160
3168099181,5cyclictest0-21swapper/020:55:150
3168099181,4cyclictest0-21swapper/023:41:070
31680991813,3cyclictest7632-21/usr/sbin/munin19:25:190
31680991813,3cyclictest7632-21/usr/sbin/munin19:25:180
3168099171,4cyclictest4621-21/usr/sbin/munin19:20:180
31680991712,3cyclictest18144-21df_inode19:50:130
151462171,3sleep00-21swapper/019:42:100
3168099166,3cyclictest2428-21/usr/sbin/munin19:15:130
3168099161,2cyclictest0-21swapper/019:36:080
31680991612,3cyclictest0-21swapper/021:09:190
3168099161,14cyclictest0-21swapper/020:21:240
3168099161,12cyclictest24410-21/usr/sbin/munin20:05:240
3168099160,14cyclictest0-21swapper/019:55:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional