You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 20:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Apr 02, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2300899290,28cyclictest0-21swapper/212:08:392
2300899270,22cyclictest524-21sh10:05:372
2301099260,25cyclictest15407-21ssh10:52:133
23008992620,5cyclictest8002-21diskmemload11:47:222
2300899260,25cyclictest8002-21diskmemload10:53:192
2300899260,25cyclictest20156-21ssh11:59:442
23003992619,6cyclictest6896-21sh11:15:001
2301099251,23cyclictest34-21ksoftirqd/309:47:053
23008992519,5cyclictest0-21swapper/211:53:352
23008992518,6cyclictest25993-21cat08:40:012
2300899250,24cyclictest8002-21diskmemload12:36:382
2300899250,24cyclictest0-21swapper/212:21:242
2300899250,24cyclictest0-21swapper/209:44:132
2300899250,0cyclictest0-21swapper/211:40:072
2300899250,0cyclictest0-21swapper/209:51:422
23003992523,1cyclictest0-21swapper/109:38:541
23003992520,5cyclictest0-21swapper/110:05:011
23003992520,4cyclictest0-21swapper/111:52:051
2300399250,24cyclictest8002-21diskmemload09:29:031
2300399250,24cyclictest0-21swapper/112:33:471
2300399250,24cyclictest0-21swapper/110:56:311
2300399250,24cyclictest0-21swapper/110:27:151
2300399250,24cyclictest0-21swapper/109:34:461
2300399250,24cyclictest0-21swapper/108:20:011
2300399250,21cyclictest9971-21ssh11:49:461
2300399250,1cyclictest0-21swapper/110:01:441
2300399250,0cyclictest0-21swapper/110:47:421
22998992525,0cyclictest0-21swapper/011:05:060
22998992523,1cyclictest0-21swapper/009:35:000
2299899251,1cyclictest13033-21sh11:21:200
2299899250,24cyclictest1721-21ssh09:35:050
2299899250,24cyclictest0-21swapper/010:35:270
2299899250,0cyclictest0-21swapper/009:51:030
2301099240,23cyclictest8002-21diskmemload11:18:083
2301099240,23cyclictest31770-21ssh10:36:043
23008992421,3cyclictest0-21swapper/210:36:202
23008992420,4cyclictest0-21swapper/211:21:402
23008992420,3cyclictest0-21swapper/211:15:102
23008992420,3cyclictest0-21swapper/209:18:362
2300899240,23cyclictest8002-21diskmemload11:12:112
2300899240,23cyclictest0-21swapper/209:55:162
2300899240,23cyclictest0-21swapper/209:34:422
2300899240,0cyclictest0-21swapper/212:03:022
2300899240,0cyclictest0-21swapper/211:31:442
2300899240,0cyclictest0-21swapper/211:27:342
2300899240,0cyclictest0-21swapper/210:18:122
2300899240,0cyclictest0-21swapper/210:13:232
2300899240,0cyclictest0-21swapper/209:36:502
2300899240,0cyclictest0-21swapper/209:10:272
23003992424,0cyclictest0-21swapper/109:54:201
23003992421,3cyclictest0-21swapper/109:45:431
23003992420,3cyclictest0-21swapper/111:55:161
23003992420,3cyclictest0-21swapper/109:19:321
2300399240,3cyclictest11565-21ls08:10:001
2300399240,24cyclictest0-21swapper/112:23:231
2300399240,24cyclictest0-21swapper/110:51:331
2300399240,24cyclictest0-21swapper/110:20:061
2300399240,23cyclictest0-21swapper/112:18:121
2300399240,23cyclictest0-21swapper/112:10:221
2300399240,23cyclictest0-21swapper/110:39:281
2300399240,21cyclictest8002-21diskmemload10:11:401
2300399240,1cyclictest0-21swapper/110:42:471
2300399240,0cyclictest0-21swapper/112:38:101
2300399240,0cyclictest0-21swapper/111:09:351
2300399240,0cyclictest0-21swapper/111:01:181
22998992423,0cyclictest0-21swapper/011:13:350
2299899240,24cyclictest0-21swapper/010:03:120
2299899240,23cyclictest8002-21diskmemload11:32:000
2299899240,23cyclictest8002-21diskmemload10:50:560
2299899240,23cyclictest0-21swapper/010:06:260
2299899240,1cyclictest0-21swapper/012:23:290
2299899240,0cyclictest0-21swapper/010:40:310
23010992323,0cyclictest8002-21diskmemload12:35:083
23010992323,0cyclictest8002-21diskmemload09:11:353
23010992323,0cyclictest0-21swapper/312:24:083
23010992323,0cyclictest0-21swapper/310:25:293
23010992322,1cyclictest23913-21ssh11:32:303
23010992321,1cyclictest3131-21ssh11:11:493
2301099231,22cyclictest0-21swapper/311:59:503
2301099231,21cyclictest331rcuc/312:10:293
2301099231,21cyclictest331rcuc/309:27:023
2301099231,21cyclictest19218-21ssh10:55:093
2301099230,2cyclictest24232-21/usr/sbin/munin09:58:423
2301099230,23cyclictest0-21swapper/312:16:253
2301099230,23cyclictest0-21swapper/310:22:523
2301099230,23cyclictest0-21swapper/310:01:083
2301099230,22cyclictest8002-21diskmemload12:07:153
2301099230,22cyclictest606-21dbus-daemon08:40:013
2301099230,22cyclictest24984-21ssh11:01:093
2301099230,22cyclictest21047-21ssh12:33:113
2301099230,22cyclictest19565-21sh09:21:523
2301099230,22cyclictest1904-21sh09:35:203
2301099230,22cyclictest15340-21ssh09:18:143
2301099230,22cyclictest1128-21ssh10:06:283
2301099230,22cyclictest0-21swapper/309:44:003
2301099230,0cyclictest8002-21diskmemload10:17:123
2301099230,0cyclictest0-21swapper/311:06:083
23008992323,0cyclictest0-21swapper/210:56:082
23008992323,0cyclictest0-21swapper/210:03:552
23008992323,0cyclictest0-21swapper/209:21:482
23008992321,1cyclictest739-21turbostat.cron08:55:002
23008992320,3cyclictest8002-21diskmemload11:05:402
23008992320,3cyclictest0-21swapper/210:26:212
23008992320,3cyclictest0-21swapper/210:22:552
23008992320,3cyclictest0-21swapper/209:25:482
23008992319,3cyclictest0-21swapper/210:47:312
2300899230,22cyclictest19659-21ssh12:30:552
2300899230,22cyclictest0-21swapper/212:27:152
2300899230,0cyclictest0-21swapper/210:31:572
23003992323,0cyclictest0-21swapper/111:31:031
23003992323,0cyclictest0-21swapper/109:20:301
23003992321,1cyclictest15033-21sh10:19:071
23003992320,3cyclictest0-21swapper/111:40:071
23003992319,0cyclictest0-21swapper/112:04:221
23003992318,1cyclictest0-21swapper/111:16:341
2300399230,23cyclictest0-21swapper/111:29:201
2300399230,22cyclictest24104-21apt-get08:38:341
2300399230,22cyclictest12887-21chrt09:14:471
2300399230,20cyclictest8002-21diskmemload12:26:001
2300399230,0cyclictest8002-21diskmemload11:21:181
2300399230,0cyclictest0-21swapper/112:05:491
22998992323,0cyclictest0-21swapper/011:47:300
22998992323,0cyclictest0-21swapper/009:21:110
22998992323,0cyclictest0-21swapper/009:16:330
22998992322,1cyclictest0-21swapper/011:27:360
22998992321,1cyclictest16354-21idleruntime-cro08:20:010
22998992321,1cyclictest0-21swapper/012:16:130
22998992320,2cyclictest21753-21ssh12:01:590
22998992320,2cyclictest20804-21sh12:32:490
22998992320,2cyclictest10267-21sh10:47:100
2299899231,1cyclictest62550irq/127-eno109:49:040
2299899230,2cyclictest32754-21sh11:40:160
2299899230,23cyclictest0-21swapper/012:26:530
2299899230,23cyclictest0-21swapper/010:22:540
2299899230,22cyclictest8002-21diskmemload11:15:030
2299899230,22cyclictest7075-21ssh09:40:230
2299899230,22cyclictest0-21swapper/012:36:010
2299899230,22cyclictest0-21swapper/010:15:060
2299899230,22cyclictest0-21swapper/009:28:410
2299899230,1cyclictest9754-21sh10:13:450
2299899230,0cyclictest8002-21diskmemload11:55:260
2299899230,0cyclictest0-21swapper/011:52:070
2299899230,0cyclictest0-21swapper/011:37:340
23010992222,0cyclictest0-21swapper/311:40:083
23010992221,1cyclictest17415-21ssh09:50:493
23010992220,1cyclictest26051-21munin-run07:20:003
23010992220,1cyclictest10245-21sh10:47:093
23010992220,1cyclictest0-21swapper/311:25:303
2301099220,2cyclictest6612-21ssh10:11:303
2301099220,22cyclictest8002-21diskmemload11:20:223
2301099220,21cyclictest8002-21diskmemload12:25:093
2301099220,21cyclictest8002-21diskmemload11:47:003
2301099220,21cyclictest28368-21apt08:45:003
2301099220,21cyclictest20550-21rm12:00:043
2301099220,21cyclictest0-21swapper/311:36:053
2301099220,21cyclictest0-21swapper/310:30:583
2301099220,21cyclictest0-21swapper/307:58:303
2301099220,1cyclictest4263-21ssh10:40:383
2301099220,1cyclictest28969-21ssh09:30:033
2301099220,1cyclictest16369-21perf08:20:013
23008992222,0cyclictest0-21swapper/211:36:212
23008992222,0cyclictest0-21swapper/209:47:022
23008992220,1cyclictest11540-21cron08:10:002
23008992219,3cyclictest0-21swapper/211:00:132
2300899220,22cyclictest0-21swapper/210:40:252
2300899220,21cyclictest837-21apt-get07:40:022
2300899220,21cyclictest30133-21munin-run07:35:012
2300899220,21cyclictest0-21swapper/212:15:512
2300899220,21cyclictest0-21swapper/208:05:002
2300899220,1cyclictest0-21swapper/212:11:552
23003992220,1cyclictest30750-21apt-get07:38:311
2300399220,21cyclictest16907-21apt-get08:23:351
2300399220,21cyclictest1260-21grep07:45:011
2300399220,21cyclictest0-21swapper/108:58:351
2300399220,21cyclictest0-21swapper/107:58:301
2300399220,1cyclictest0-21swapper/109:55:141
22998992222,0cyclictest0-21swapper/009:55:510
22998992221,1cyclictest25492-21ssh11:02:030
22998992221,0cyclictest0-21swapper/012:11:550
22998992221,0cyclictest0-21swapper/008:23:350
22998992220,2cyclictest0-21swapper/012:05:030
2299899220,21cyclictest8002-21diskmemload10:25:210
2299899220,21cyclictest0-21swapper/010:55:150
2299899220,21cyclictest0-21swapper/007:40:020
2299899220,1cyclictest0-21swapper/009:13:350
2299899220,1cyclictest0-21swapper/008:15:000
2299899220,1cyclictest0-21swapper/007:28:300
23010992120,0cyclictest0-21swapper/308:08:333
2301099210,20cyclictest0-21swapper/309:08:343
2301099210,20cyclictest0-21swapper/308:13:343
2301099210,0cyclictest0-21swapper/308:33:343
23008992120,1cyclictest14491-21apt-get08:18:342
23008992119,1cyclictest30791-21munin-run08:50:002
2300899210,20cyclictest0-21swapper/209:03:362
2300899210,20cyclictest0-21swapper/207:38:312
23003992120,1cyclictest7305-21apt-get08:03:351
23003992119,1cyclictest0-21swapper/108:35:011
2300399210,0cyclictest0-21swapper/108:43:351
22998992121,0cyclictest0-21swapper/010:31:010
22998992120,1cyclictest5580-21turbostat09:05:000
22998992119,1cyclictest7998-21cron09:10:010
22998992119,1cyclictest25989-21sendmail-msp08:40:010
22998992119,1cyclictest23621-21sh07:15:000
2299899210,1cyclictest0-21swapper/008:00:000
2299899210,0cyclictest0-21swapper/007:38:310
2299899210,0cyclictest0-21swapper/007:18:310
23010992020,0cyclictest0-21swapper/309:03:353
23010992020,0cyclictest0-21swapper/307:38:313
23008992020,0cyclictest28383-21perf08:45:002
23008992018,1cyclictest3188-21awk09:00:002
2300899200,19cyclictest0-21swapper/208:33:342
2300899200,0cyclictest0-21swapper/208:23:352
23003992018,1cyclictest21148-21cron08:30:011
2300399200,1cyclictest0-21swapper/107:30:011
22998992020,0cyclictest0-21swapper/008:33:340
22998992020,0cyclictest0-21swapper/008:28:340
23010991919,0cyclictest0-21swapper/308:03:353
23010991916,2cyclictest19864-21gltestperf08:28:363
23010991915,3cyclictest29107-21grep08:48:353
22998991918,1cyclictest28913-21apt-get08:48:340
22998991917,1cyclictest3161-21latency_hist09:00:000
23010991818,0cyclictest0-21swapper/308:53:353
23008991818,0cyclictest6104-21apt-get09:08:342
23003991818,0cyclictest24153-21apt-get07:18:301
23003991818,0cyclictest0-21swapper/108:13:341
23003991817,0cyclictest0-21swapper/108:48:341
23003991816,1cyclictest4277-21cron07:55:001
23003991816,1cyclictest0-21swapper/107:35:011
23003991816,1cyclictest0-21swapper/107:25:001
22998991818,0cyclictest0-21swapper/008:08:330
23010991717,0cyclictest0-21swapper/307:28:303
23010991714,2cyclictest0-21swapper/307:15:003
23008991717,0cyclictest0-21swapper/208:13:342
23003991716,1cyclictest3710-21apt-get09:03:361
22998991717,0cyclictest0-21swapper/007:48:300
23010991614,1cyclictest18809-21sed08:25:013
23008991616,0cyclictest0-21swapper/207:48:302
2300899160,0cyclictest0-21swapper/207:54:222
23003991615,0cyclictest0-21swapper/109:08:341
22998991616,0cyclictest0-21swapper/008:03:350
2299899160,15cyclictest0-21swapper/008:43:430
23008991514,0cyclictest0-21swapper/207:58:302
2300899150,1cyclictest0-21swapper/207:28:322
23003991515,0cyclictest0-21swapper/107:50:011
22998991515,0cyclictest31311-21apt-get08:53:350
2301099140,14cyclictest0-21swapper/307:50:093
2301099140,14cyclictest0-21swapper/307:20:293
2301099140,0cyclictest0-21swapper/307:40:463
2301099140,0cyclictest0-21swapper/307:30:453
2300899140,14cyclictest0-21swapper/207:20:552
2300899140,14cyclictest0-21swapper/207:16:102
2300899140,0cyclictest0-21swapper/208:25:352
2300899140,0cyclictest0-21swapper/207:12:292
22998991414,0cyclictest0-21swapper/007:50:110
2299899140,14cyclictest0-21swapper/007:30:080
2299899140,14cyclictest0-21swapper/007:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional