You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 00:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Dec 10, 2025 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2653922300,2sleep226545-21kworker/u8:007:10:152
266702950,4sleep02680699cyclictest11:22:280
26806999216,13cyclictest7850irq/56-dwc_otg_10:45:300
47502860,3sleep34745-21ssh11:37:413
26806998224,29cyclictest8646-21sshd10:55:280
26806998224,29cyclictest8646-21sshd10:55:280
191272800,5sleep319130-21unixbench_singl11:10:443
26806997815,14cyclictest26805-21cyclictest11:05:280
26806997734,14cyclictest0-21swapper/007:55:280
26809997047,19cyclictest1156-21sshd11:55:283
26809997047,19cyclictest1156-21sshd11:55:283
2680699681,40cyclictest30720-21fschecks_count07:20:350
2680699676,44cyclictest11680-21sh11:00:280
26806996714,34cyclictest9-21ksoftirqd/009:35:270
2680699669,10cyclictest140-21systemd-journal11:15:160
239932660,2sleep10-21swapper/109:38:431
26806996319,27cyclictest3904-21grep11:35:440
2680699622,6cyclictest9191-21ssh12:30:470
2680699622,6cyclictest9191-21ssh12:30:470
2680699622,32cyclictest0-21swapper/012:30:170
2680699615,37cyclictest9-21ksoftirqd/009:10:460
26806996118,17cyclictest7950irq/56-dwc_otg_11:55:310
26806996118,17cyclictest7950irq/56-dwc_otg_11:55:310
2680699611,34cyclictest30099-21sed08:45:470
26806996030,15cyclictest7950irq/56-dwc_otg_09:50:220
26806996027,18cyclictest7750irq/56-dwc_otg10:50:230
2680999596,10cyclictest7750irq/56-dwc_otg12:20:273
26807995916,19cyclictest7850irq/56-dwc_otg_10:30:301
26807995915,25cyclictest14525-21ssh09:24:591
126682590,2sleep112671-21aten_rbpower_cu08:00:291
126682590,2sleep112671-21aten_rbpower_cu08:00:291
26807995832,8cyclictest813-21apt-get12:20:291
26807995829,23cyclictest26810-21chrt07:10:181
26807995829,23cyclictest26810-21chrt07:10:181
26806995844,5cyclictest7750irq/56-dwc_otg08:25:150
2680699582,34cyclictest0-21swapper/009:30:310
26806995822,22cyclictest0-21swapper/007:10:180
26806995822,22cyclictest0-21swapper/007:10:180
73732570,2sleep00-21swapper/010:00:480
26806995732,12cyclictest7750irq/56-dwc_otg10:00:160
26806995724,6cyclictest7950irq/56-dwc_otg_12:25:160
26806995717,4cyclictest29710-21unixbench_multi10:35:520
2680699571,39cyclictest0-21swapper/008:50:410
2680699571,39cyclictest0-21swapper/008:50:410
2680999565,32cyclictest0-21swapper/307:10:183
2680999565,32cyclictest0-21swapper/307:10:183
2680999561,3cyclictest0-21swapper/312:36:493
26807995621,20cyclictest10073-21ssh10:56:321
26807995621,20cyclictest10073-21ssh10:56:321
26807995613,10cyclictest7850irq/56-dwc_otg_07:50:301
26807995612,26cyclictest10556-21ssh10:06:081
2680699566,41cyclictest0-21swapper/012:15:300
119342560,5sleep02680699cyclictest09:20:310
26809995510,3cyclictest0-21swapper/310:39:353
26807995512,27cyclictest22886-21ntp_states10:25:401
2680699556,46cyclictest0-21swapper/012:00:290
26806995542,4cyclictest7850irq/56-dwc_otg_12:05:360
26806995523,5cyclictest32621-21aten_rbpower_en08:55:350
26806995523,5cyclictest32621-21aten_rbpower_en08:55:350
2680699552,28cyclictest28696-21cron08:45:160
2680699552,25cyclictest0-21swapper/008:30:270
26806995511,26cyclictest6457-21unixbench_multi07:40:450
2680999542,19cyclictest0-21swapper/309:52:243
26807995422,18cyclictest27622-21ssh09:44:191
26807995417,23cyclictest18868-21apt10:20:271
2680699542,31cyclictest22851-21unixbench_singl08:25:440
2680699541,39cyclictest28989-21ssh12:14:020
193192540,4sleep319307-21ssh10:20:363
2680999538,6cyclictest25314-21kernelversion11:20:373
2680799534,13cyclictest7750irq/56-dwc_otg10:40:291
26807995323,15cyclictest7234-21/usr/sbin/munin07:45:321
26807995322,18cyclictest17049-21ssh09:27:491
26807995316,21cyclictest0-21swapper/112:32:031
26807995316,21cyclictest0-21swapper/112:32:031
26807995316,13cyclictest6663-21/usr/sbin/munin10:50:411
26807995313,25cyclictest22857-21taskset11:16:171
26807995313,23cyclictest1425-21rm09:52:381
2680699536,27cyclictest9379-21latency_hist11:45:160
26806995335,5cyclictest7950irq/56-dwc_otg_08:35:300
26806995329,6cyclictest7750irq/56-dwc_otg09:18:260
26806995323,8cyclictest7850irq/56-dwc_otg_10:25:280
26806995316,15cyclictest28022-21sh10:34:460
2680699531,43cyclictest14578-21kworker/0:110:17:210
2680699531,33cyclictest0-21swapper/010:44:160
317652520,1sleep30-21swapper/312:17:263
26807995221,18cyclictest30556-21ssh10:37:491
2680699524,30cyclictest21960-21cat11:15:370
26806995230,4cyclictest7850irq/56-dwc_otg_11:25:440
2680699522,6cyclictest0-21swapper/009:25:260
2680699522,31cyclictest0-21swapper/007:25:450
2680699522,28cyclictest401-21unixbench_multi11:30:440
2680699522,27cyclictest0-21swapper/010:05:490
26806995220,15cyclictest7750irq/56-dwc_otg08:10:450
2680699521,33cyclictest9724-21ntp_kernel_pll_07:50:430
26809995135,6cyclictest7850irq/56-dwc_otg_09:35:273
26807995119,14cyclictest16394-21ssh11:55:271
26807995119,14cyclictest16394-21ssh11:55:271
26807995116,12cyclictest14903-21ssh11:52:171
26807995115,23cyclictest13703-21/usr/sbin/munin10:10:551
26807995115,12cyclictest2971-21taskset11:35:331
26807995114,16cyclictest31363-21ls11:30:151
2680699512,3cyclictest24933-21sshd09:40:270
26806995122,17cyclictest0-21swapper/012:35:530
66252500,1sleep30-21swapper/312:28:053
2680999507,11cyclictest0-21swapper/310:30:333
2680899504,24cyclictest0-21swapper/209:00:292
26808995011,12cyclictest813-21apt-get12:20:272
26807995014,24cyclictest14433-21rm12:39:301
26807995014,14cyclictest5297-21diskmemload11:20:271
26807995014,13cyclictest20649-21ssh12:00:431
26807995012,25cyclictest3273-21taskset09:02:521
26807995011,12cyclictest31331-21snmpd12:06:481
2680699502,30cyclictest14591-21sh11:51:290
2680999497,15cyclictest0-21swapper/311:15:313
2680799499,9cyclictest0-21swapper/109:58:071
2680799499,15cyclictest30358-21ssh09:47:301
2680799498,13cyclictest6420-21fschecks_time10:00:331
26807994917,7cyclictest12452-21ssh11:49:321
26807994913,12cyclictest18746-21if_eth009:30:341
26807994911,14cyclictest11854-21cat07:55:441
2680699497,23cyclictest13219-21/usr/sbin/munin10:10:500
2680699492,26cyclictest13-21rcuc/008:00:420
2680699492,26cyclictest13-21rcuc/008:00:420
2680899483,27cyclictest0-21swapper/209:55:312
26807994816,21cyclictest5690-21ssh12:25:571
26807994815,14cyclictest23289-21latency_hist08:30:161
26807994814,12cyclictest18551-21missed_timers11:10:371
26806994818,16cyclictest0-21swapper/009:00:310
2680999477,26cyclictest0-21swapper/308:35:303
2680899477,25cyclictest0-21swapper/212:00:312
2680799479,13cyclictest31592-21ssh11:30:181
26807994721,12cyclictest3940-21diskstats09:05:321
26807994712,12cyclictest6903-21apt07:45:151
26807994711,14cyclictest26143-21/usr/sbin/munin08:35:421
26806994716,22cyclictest0-21swapper/009:06:560
2680699471,3cyclictest0-21swapper/007:15:280
2680999467,25cyclictest0-21swapper/308:40:323
2680999461,16cyclictest0-21swapper/309:30:333
26808994629,9cyclictest7850irq/56-dwc_otg_08:30:432
26808994617,19cyclictest7950irq/56-dwc_otg_09:32:272
2680799468,12cyclictest7750irq/56-dwc_otg12:15:161
2680799467,24cyclictest0-21swapper/107:35:311
2680799467,24cyclictest0-21swapper/107:35:311
26807994614,11cyclictest24420-21timerandwakeup08:30:421
26807994612,23cyclictest16648-21taskset11:07:231
26807994611,16cyclictest14295-21ssh11:04:351
2680699462,22cyclictest0-21swapper/011:45:290
2680699461,6cyclictest0-21swapper/008:15:290
26806994614,14cyclictest7850irq/56-dwc_otg_08:05:300
2680999457,24cyclictest0-21swapper/312:30:303
2680999457,24cyclictest0-21swapper/312:30:303
2680799452,3cyclictest0-21swapper/108:10:281
26807994511,21cyclictest7984-21sh09:14:091
26807994510,10cyclictest7750irq/56-dwc_otg11:43:561
26806994520,14cyclictest3814-21taskset07:35:350
26806994520,14cyclictest3814-21taskset07:35:350
26806994514,7cyclictest7750irq/56-dwc_otg07:50:150
2680699451,20cyclictest0-21swapper/009:46:300
2680999447,24cyclictest0-21swapper/307:25:213
2680899447,23cyclictest0-21swapper/211:00:222
2680899444,19cyclictest7950irq/56-dwc_otg_11:35:342
26808994423,16cyclictest26805-21cyclictest07:10:172
26808994423,16cyclictest26805-21cyclictest07:10:172
2680899441,5cyclictest0-21swapper/212:25:282
26808994411,15cyclictest7950irq/56-dwc_otg_07:55:442
2680799449,14cyclictest0-21swapper/107:24:291
2680799448,24cyclictest0-21swapper/108:50:301
2680799448,24cyclictest0-21swapper/108:50:301
26807994421,11cyclictest10256-21sshd09:16:281
26807994418,13cyclictest2696-21rm10:45:231
26807994411,9cyclictest0-21swapper/112:16:581
2680699446,21cyclictest9-21ksoftirqd/010:25:120
264472443,34sleep00-21swapper/007:09:090
2680899437,23cyclictest0-21swapper/210:40:312
2680899437,22cyclictest0-21swapper/212:30:232
2680899437,22cyclictest0-21swapper/212:30:232
2680899431,8cyclictest0-21swapper/210:45:362
26808994313,8cyclictest0-21swapper/210:55:302
26808994313,8cyclictest0-21swapper/210:55:302
2680799438,17cyclictest7950irq/56-dwc_otg_08:06:381
26807994313,12cyclictest21599-21sed08:25:161
2680999427,21cyclictest0-21swapper/311:50:323
2680999427,21cyclictest0-21swapper/308:15:323
2680999426,25cyclictest9049-21seq07:50:303
2680899427,26cyclictest0-21swapper/211:20:302
2680899427,23cyclictest0-21swapper/209:45:302
2680899427,23cyclictest0-21swapper/208:10:292
2680899426,24cyclictest0-21swapper/210:15:322
2680799428,21cyclictest7750irq/56-dwc_otg07:29:391
26807994212,19cyclictest17477-21ssh10:17:161
2680999417,24cyclictest0-21swapper/309:10:293
2680899417,26cyclictest16419-21sshd11:55:282
2680899417,26cyclictest16419-21sshd11:55:282
2680899417,24cyclictest0-21swapper/209:40:302
2680899417,22cyclictest0-21swapper/209:50:222
2680899417,21cyclictest0-21swapper/207:35:312
2680899417,21cyclictest0-21swapper/207:35:312
26808994115,9cyclictest7950irq/56-dwc_otg_10:20:322
26808994115,5cyclictest28656-21aten_rbpower_po10:35:342
255502415,23sleep125642-21/usr/sbin/munin07:05:381
2680999407,24cyclictest0-21swapper/307:40:303
2680999406,25cyclictest0-21swapper/309:00:313
26809994018,5cyclictest3713-21acpi09:05:263
26808994024,9cyclictest7750irq/56-dwc_otg09:30:162
2680799408,17cyclictest0-21swapper/109:00:111
2680799408,17cyclictest0-21swapper/109:00:111
2680799408,13cyclictest0-21swapper/108:49:351
2680799404,6cyclictest7950irq/56-dwc_otg_08:40:301
26807994011,14cyclictest29899-21seq07:16:411
2680999397,20cyclictest0-21swapper/307:15:313
2680999391,4cyclictest7950irq/56-dwc_otg_10:50:163
2680899397,16cyclictest0-21swapper/208:50:302
2680899397,16cyclictest0-21swapper/208:50:302
2680899391,23cyclictest18233-21ls08:15:342
2680899391,23cyclictest0-21swapper/207:50:212
2680799396,10cyclictest7850irq/56-dwc_otg_08:19:371
2680699392,21cyclictest0-21swapper/007:32:260
2680999381,27cyclictest9829-21ls10:05:483
2680999381,25cyclictest31339-21missed_timers08:50:373
2680999381,25cyclictest31339-21missed_timers08:50:373
2680899389,9cyclictest6069-21apt-get10:00:282
2680899386,12cyclictest1393-21ssh11:32:452
2680899386,11cyclictest7950irq/56-dwc_otg_11:10:402
26808993815,6cyclictest7750irq/56-dwc_otg11:30:152
26808993814,10cyclictest7750irq/56-dwc_otg09:24:432
26808993813,11cyclictest7950irq/56-dwc_otg_12:06:242
2680999376,22cyclictest0-21swapper/309:55:313
2680999372,5cyclictest7950irq/56-dwc_otg_07:35:303
2680999372,5cyclictest7950irq/56-dwc_otg_07:35:303
2680999371,22cyclictest0-21swapper/311:00:213
2680899377,24cyclictest0-21swapper/208:55:322
2680899377,24cyclictest0-21swapper/208:55:322
26808993715,5cyclictest8824-21sh11:44:212
26808993714,5cyclictest21712-21fschecks_time11:15:352
82192360,6sleep32680999cyclictest10:03:213
2680999364,12cyclictest7850irq/56-dwc_otg_08:45:313
2680899365,19cyclictest7950irq/56-dwc_otg_11:52:432
26808993620,8cyclictest7750irq/56-dwc_otg12:37:372
26808993612,9cyclictest6988-21ssh09:11:002
26808993610,13cyclictest7950irq/56-dwc_otg_10:30:302
2680999357,15cyclictest0-21swapper/310:50:223
2680999351,23cyclictest9014-21ls09:15:333
2680899357,4cyclictest6899-21idleruntime-cro07:45:152
2680899357,19cyclictest0-21swapper/210:50:232
2680899357,18cyclictest0-21swapper/208:35:302
2680899355,13cyclictest301ktimersoftd/210:25:342
2680899351,24cyclictest17666-21ls11:10:162
2680899351,18cyclictest7950irq/56-dwc_otg_09:05:372
26808993510,10cyclictest7850irq/56-dwc_otg_12:12:022
2680999346,14cyclictest0-21swapper/307:30:323
2680999344,6cyclictest7850irq/56-dwc_otg_11:05:453
2680899349,8cyclictest11759-21ssh10:09:152
2680899346,11cyclictest31-21ksoftirqd/208:01:162
2680899346,11cyclictest31-21ksoftirqd/208:01:162
2680899343,17cyclictest7950irq/56-dwc_otg_09:19:382
2680899342,13cyclictest31-21ksoftirqd/210:14:502
26808993419,4cyclictest519-21munin-node08:05:262
26808993415,7cyclictest7950irq/56-dwc_otg_07:45:302
2680899341,21cyclictest27497-21ls08:40:372
26808993412,13cyclictest7850irq/56-dwc_otg_09:35:432
2680899341,20cyclictest0-21swapper/212:16:592
2680999339,5cyclictest2463-21latency_hist11:35:163
2680999339,11cyclictest7950irq/56-dwc_otg_08:25:383
2680999337,5cyclictest27188-21missed_timers12:10:373
2680999337,24cyclictest0-21swapper/307:45:313
2680999333,16cyclictest22488-21fschecks_time10:25:343
2680999332,14cyclictest7850irq/56-dwc_otg_10:13:303
2680999331,25cyclictest8944-21uniq10:55:373
2680999331,25cyclictest8944-21uniq10:55:373
2680999331,16cyclictest7750irq/56-dwc_otg11:45:453
2680899331,18cyclictest0-21swapper/207:25:162
2680799336,10cyclictest0-21swapper/107:34:481
2680999326,11cyclictest7950irq/56-dwc_otg_11:25:373
2680999321,22cyclictest28047-21latency_hist09:45:163
2680899329,5cyclictest29-21rcuc/208:30:162
2680899328,9cyclictest7850irq/56-dwc_otg_11:47:052
2680999316,5cyclictest7750irq/56-dwc_otg08:10:363
2680999316,14cyclictest28356-21sshd09:45:283
2680999311,9cyclictest16146-21sed09:25:453
2680999311,20cyclictest15628-21ssh10:15:323
2680899312,14cyclictest7950irq/56-dwc_otg_08:22:372
2680899311,17cyclictest2024-21fschecks_time07:30:362
2680999307,14cyclictest0-21swapper/308:30:293
2680999303,5cyclictest7950irq/56-dwc_otg_11:40:433
2680999301,5cyclictest13060-21unixbench_multi09:20:513
26809993012,5cyclictest2439-21ls10:45:163
2680999301,23cyclictest0-21swapper/312:02:583
2680899301,26cyclictest514-21ls07:25:392
2680999298,5cyclictest31578-21grep07:20:463
2680999296,4cyclictest7850irq/56-dwc_otg_12:05:273
2680999282,11cyclictest10840-21/usr/sbin/munin07:55:393
2680999281,13cyclictest7950irq/56-dwc_otg_08:56:533
2680999281,13cyclictest7950irq/56-dwc_otg_08:56:533
2680999275,4cyclictest12648-21/usr/sbin/munin08:00:353
2680999275,4cyclictest12648-21/usr/sbin/munin08:00:353
2680999272,18cyclictest0-21swapper/308:06:543
2680899277,17cyclictest0-21swapper/207:15:312
2680899271,17cyclictest0-21swapper/208:47:292
256802277,11sleep30-21swapper/307:05:403
2680999252,5cyclictest7850irq/56-dwc_otg_08:20:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional