You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 09:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 11, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1808921940,2sleep30-21swapper/304:26:013
19417998924,34cyclictest7750irq/56-dwc_otg08:25:531
140632880,5sleep21941899cyclictest05:45:412
203572870,2sleep20-21swapper/208:12:432
64442840,8sleep3381ktimersoftd/309:53:333
1941699845,20cyclictest17229-21ssh07:30:470
1941699813,46cyclictest0-21swapper/009:55:540
19417997810,12cyclictest7750irq/56-dwc_otg07:35:531
163172780,5sleep01941699cyclictest05:50:580
1941699761,9cyclictest0-21swapper/008:50:540
19416997428,9cyclictest0-21swapper/004:50:500
104472720,1sleep10-21swapper/109:58:181
75632710,2sleep17112-21/usr/sbin/munin09:15:591
78002700,2sleep20-21swapper/207:57:292
19418997020,23cyclictest7153-21sh09:15:542
1941899701,57cyclictest24370-21/usr/sbin/munin08:56:102
1941899696,12cyclictest7850irq/56-dwc_otg_06:50:552
1941899696,12cyclictest7850irq/56-dwc_otg_06:50:552
1941699695,41cyclictest0-21swapper/005:10:520
1941699695,41cyclictest0-21swapper/005:10:520
1941899681,52cyclictest2911-21/usr/sbin/munin08:31:012
1941899671,53cyclictest315-21ssh09:06:272
1941699667,43cyclictest0-21swapper/004:40:530
1941699666,29cyclictest9-21ksoftirqd/009:15:410
1941899651,47cyclictest10192-21ssh06:44:472
1941899651,39cyclictest495250ssh07:54:522
1941699653,31cyclictest18173-21ssh08:10:510
19416996529,5cyclictest0-21swapper/008:35:510
212002640,2sleep10-21swapper/107:35:381
1941899641,51cyclictest14924-21ssh06:50:362
1941899641,50cyclictest22262-21ssh08:54:512
1941899641,50cyclictest10839-21ssh09:59:022
1941899641,45cyclictest8979-21df_inode05:30:572
1941899641,42cyclictest13781-21sed08:05:412
19416996417,22cyclictest908-21runrttasks06:46:160
165222640,2sleep10-21swapper/109:26:091
49182630,1sleep00-21swapper/006:38:040
1941899633,39cyclictest14526-21ssh08:06:012
1941899631,52cyclictest26924-21ssh07:04:112
1941899631,43cyclictest24064-21ssh08:16:542
1941899621,49cyclictest16920-21ssh09:26:302
1941899621,49cyclictest13640-21ssh09:23:052
1941899621,42cyclictest5709-21ssh09:52:132
1941899621,42cyclictest1239-21ssh09:46:372
1941899621,40cyclictest5349-21ssh09:13:032
1941899621,34cyclictest0-21swapper/208:25:412
1941799628,11cyclictest7950irq/56-dwc_otg_05:10:521
1941799628,11cyclictest7950irq/56-dwc_otg_05:10:521
1941699621,48cyclictest0-21swapper/006:51:110
1941699621,48cyclictest0-21swapper/006:51:110
1941899611,40cyclictest20396-21ntp_states06:56:042
1941899611,37cyclictest31270-21/usr/sbin/munin06:31:012
1941899611,35cyclictest26005-21df_inode07:40:572
1941699614,32cyclictest13-21rcuc/005:01:010
1941699612,32cyclictest20876-21sed06:56:090
1941699611,29cyclictest0-21swapper/006:25:540
190932610,13sleep10-21swapper/104:30:011
19418996039,13cyclictest908-21runrttasks05:20:532
1941899602,42cyclictest26666-21apt-get04:50:502
1941899601,45cyclictest0-21swapper/208:41:252
1941899601,43cyclictest31315-21ls05:01:092
1941899601,42cyclictest31426-21rm07:46:322
1941899601,41cyclictest26466-21chrt09:38:352
1941899601,41cyclictest26466-21chrt09:38:352
1941899600,42cyclictest14499-21ssh07:26:352
1941799604,9cyclictest7850irq/56-dwc_otg_06:45:521
1941699604,50cyclictest7950irq/56-dwc_otg_08:15:510
1941699602,38cyclictest13-21rcuc/006:06:080
1941699601,32cyclictest30219-21fschecks_count07:46:000
188842603,14sleep20-21swapper/204:27:232
1941899591,43cyclictest20610-21cat06:01:072
1941899591,39cyclictest774-21ssh08:28:302
1941899591,36cyclictest25352-21ssh07:40:342
1941799592,13cyclictest0-21swapper/104:35:551
1941699592,33cyclictest19527-21aten_rbpower_po04:30:540
1941699592,33cyclictest19527-21aten_rbpower_po04:30:540
19416995916,18cyclictest7647-21irqstats09:16:000
1941699591,29cyclictest27798-21cron07:05:410
1941999584,7cyclictest7950irq/56-dwc_otg_05:45:533
1941899586,29cyclictest7750irq/56-dwc_otg07:20:412
1941899582,38cyclictest6449-21munin-run06:40:412
19418995817,33cyclictest3914-21ssh07:15:282
19418995817,33cyclictest3914-21ssh07:15:282
1941899581,39cyclictest19215-21rm07:32:112
19416995823,17cyclictest0-21swapper/007:55:520
19416995818,21cyclictest4495-21head05:16:080
235732570,3sleep10-21swapper/108:16:081
1941899571,41cyclictest18489-21ssh08:50:322
1941899571,39cyclictest9578-21ssh08:39:222
1941899571,38cyclictest22003-21ssh09:33:012
1941899571,38cyclictest11517-21ssh07:23:482
1941699573,24cyclictest0-21swapper/006:45:420
1941699571,47cyclictest24336-21grep08:56:090
1941699571,46cyclictest0-21swapper/007:25:520
1941699571,35cyclictest25129-21fschecks_count04:45:580
1941899566,41cyclictest7950irq/56-dwc_otg_07:05:462
1941899561,36cyclictest1086-21chrt05:09:342
1941799567,8cyclictest7950irq/56-dwc_otg_05:55:541
1941799565,18cyclictest7950irq/56-dwc_otg_08:30:551
1941699565,35cyclictest0-21swapper/007:40:580
1941999557,4cyclictest0-21swapper/308:55:533
1941899551,37cyclictest28736-21apt-get06:25:522
19416995511,26cyclictest14993-21/usr/sbin/munin08:45:560
1941899541,37cyclictest29173-21unixbench_multi09:41:082
1941899541,35cyclictest30559-21ssh09:04:582
1941699545,25cyclictest0-21swapper/005:40:530
1941699543,24cyclictest10679-21apt-get05:35:520
1941699543,24cyclictest10679-21apt-get05:35:520
1941699542,30cyclictest0-21swapper/008:01:030
19416995419,5cyclictest0-21swapper/009:30:540
1941699541,29cyclictest27937-21grep09:01:020
1941699541,29cyclictest0-21swapper/009:35:520
1941699541,29cyclictest0-21swapper/009:35:520
1941899531,49cyclictest23259-21ls06:10:422
1941899531,48cyclictest23059-21apt-get04:40:522
1941899531,42cyclictest0-21swapper/205:17:472
1941899531,38cyclictest0-21swapper/205:25:522
19418995311,39cyclictest25816-21ntp_states06:16:042
19416995328,6cyclictest7850irq/56-dwc_otg_09:50:560
19416995324,8cyclictest2177-21grep07:51:010
1941699532,29cyclictest21632-21aten_rbpower_cu07:35:560
1941699532,28cyclictest0-21swapper/005:55:520
1941699532,26cyclictest27043-21aten_rbpower_vo06:20:550
1941999521,12cyclictest7850irq/56-dwc_otg_04:55:533
1941899521,47cyclictest917-21snmpd06:21:082
1941899521,46cyclictest10335-21apt-get05:35:462
1941899521,46cyclictest10335-21apt-get05:35:462
1941699524,5cyclictest10645-21ssh08:40:540
1941699522,29cyclictest15788-21cut09:26:000
19416995219,16cyclictest2661-21modprobe09:10:410
1941899511,37cyclictest29536-21grep04:56:092
1941899511,32cyclictest17437-21seq05:52:592
1941899511,31cyclictest0-21swapper/204:50:282
1941799512,6cyclictest0-21swapper/104:45:551
1941699518,27cyclictest0-21swapper/008:25:530
19416995122,16cyclictest28002-21aten_rbpower_cu09:40:540
1941899501,46cyclictest24061-21/usr/sbin/munin06:11:042
1941899491,36cyclictest0-21swapper/205:55:462
1941799497,28cyclictest0-21swapper/106:15:541
1941699496,28cyclictest0-21swapper/006:10:530
19416994930,3cyclictest0-21swapper/008:06:030
19416994928,6cyclictest7850irq/56-dwc_otg_04:40:410
1941699492,6cyclictest0-21swapper/009:25:430
19416994926,4cyclictest7750irq/56-dwc_otg09:45:540
19416994921,5cyclictest7068-21aten_rbpower_vo05:25:550
1941899481,39cyclictest0-21swapper/204:31:392
1941899481,39cyclictest0-21swapper/204:31:392
1941799484,6cyclictest0-21swapper/106:40:541
19417994810,10cyclictest1609-21ssh07:50:531
1941699481,24cyclictest0-21swapper/005:05:530
1941899471,43cyclictest22556-21cron04:37:412
1941899471,42cyclictest2074-21grep05:11:032
1941899471,42cyclictest2074-21grep05:11:032
1941899471,35cyclictest15600-21taskset05:47:342
1941699472,26cyclictest4537-21grep07:15:560
1941699472,26cyclictest0-21swapper/007:11:210
1941699472,26cyclictest0-21swapper/007:11:210
1941699472,21cyclictest0-21swapper/005:46:080
19416994713,16cyclictest26688-21sh06:20:410
1941699471,22cyclictest30615-21ssh06:30:510
1941699462,26cyclictest13-21rcuc/008:25:090
1941699461,4cyclictest19955-21uniq06:00:580
191592460,2sleep30-21swapper/307:32:063
19419994525,6cyclictest7950irq/56-dwc_otg_09:19:363
1941799457,23cyclictest0-21swapper/105:05:551
19417994524,19cyclictest338250wget05:15:541
19416994522,5cyclictest9073-21rm07:21:000
19416994519,7cyclictest5669-21grep05:21:010
188752453,12sleep00-21swapper/004:27:170
1941799442,9cyclictest0-21swapper/107:15:541
1941799442,7cyclictest0-21swapper/105:30:531
19417994424,18cyclictest2155150wget06:05:541
1941699441,27cyclictest0-21swapper/007:05:590
1941999437,23cyclictest0-21swapper/305:20:553
1941999432,10cyclictest0-21swapper/306:00:523
1941799436,27cyclictest15369-21ssh06:50:571
1941799436,27cyclictest15369-21ssh06:50:571
1941799435,10cyclictest14071-21ssh08:05:551
1941699432,22cyclictest0-21swapper/008:30:560
19416994314,14cyclictest0-21swapper/005:35:410
1941699431,21cyclictest28532-21aten_rbpower_en04:55:560
1941799429,14cyclictest221ktimersoftd/107:20:591
1941799426,22cyclictest0-21swapper/106:25:541
1941799425,21cyclictest7850irq/56-dwc_otg_08:50:541
19417994216,12cyclictest23067-21ssh08:55:531
1941999407,23cyclictest0-21swapper/309:45:553
1941999407,18cyclictest0-21swapper/305:15:473
19419994032,4cyclictest0-21swapper/307:35:553
1941999401,24cyclictest11762-21awk10:00:413
1941799407,24cyclictest0-21swapper/109:10:541
1941799407,22cyclictest0-21swapper/106:30:541
1941799404,25cyclictest0-21swapper/107:40:531
1941799404,15cyclictest7850irq/56-dwc_otg_07:15:421
1941799404,15cyclictest7850irq/56-dwc_otg_07:15:421
1941999399,4cyclictest7850irq/56-dwc_otg_04:50:413
1941999397,23cyclictest0-21swapper/308:00:543
1941999396,15cyclictest0-21swapper/308:30:553
19419993917,5cyclictest31129-21if_wlan006:30:593
1941999391,16cyclictest7950irq/56-dwc_otg_05:26:063
1941799392,21cyclictest7950irq/56-dwc_otg_09:30:541
19417993920,9cyclictest23-21ksoftirqd/104:56:031
19419993818,4cyclictest7950irq/56-dwc_otg_08:46:093
19419993814,13cyclictest27189-21ssh09:00:513
1941999379,7cyclictest7850irq/56-dwc_otg_07:11:103
1941999379,7cyclictest7850irq/56-dwc_otg_07:11:103
19419993714,5cyclictest24216-21fschecks_time07:00:583
1941999371,30cyclictest21735-21latency04:36:023
1941799378,17cyclictest0-21swapper/108:40:541
1941799371,6cyclictest29464-21rm08:24:121
19417993711,6cyclictest7750irq/56-dwc_otg07:00:551
19417993711,11cyclictest0-21swapper/109:20:551
82142360,2sleep30-21swapper/307:20:303
1941999362,5cyclictest3181250ssh08:26:083
1941799367,6cyclictest7950irq/56-dwc_otg_05:26:011
1941799367,5cyclictest2810-21tune2fs06:35:591
1941799366,16cyclictest7950irq/56-dwc_otg_05:01:091
1941799363,26cyclictest7950irq/56-dwc_otg_07:45:561
1941799361,25cyclictest28257-21ls09:01:071
19417993611,6cyclictest0-21swapper/105:20:521
1941799361,15cyclictest7950irq/56-dwc_otg_09:41:061
1941799361,14cyclictest7750irq/56-dwc_otg08:12:211
3632350,2sleep10-21swapper/109:06:331
1941999359,6cyclictest0-21swapper/309:05:593
1941999359,5cyclictest7950irq/56-dwc_otg_08:10:583
1941999359,5cyclictest7850irq/56-dwc_otg_05:40:553
1941999356,6cyclictest7950irq/56-dwc_otg_09:30:433
1941999355,15cyclictest7950irq/56-dwc_otg_08:00:423
1941999351,23cyclictest15542-21cpuspeed06:50:583
1941999351,23cyclictest15542-21cpuspeed06:50:583
19419993510,7cyclictest18900-21/usr/sbin/munin08:51:053
1941799357,5cyclictest7950irq/56-dwc_otg_09:50:511
1941799357,14cyclictest7950irq/56-dwc_otg_04:31:031
1941799357,14cyclictest7950irq/56-dwc_otg_04:31:031
1941799356,15cyclictest7950irq/56-dwc_otg_08:46:021
19417993514,5cyclictest908-21runrttasks08:37:341
1941999342,16cyclictest7950irq/56-dwc_otg_05:30:583
19419993413,5cyclictest25016-21unixbench_multi09:36:103
19419993413,5cyclictest25016-21unixbench_multi09:36:103
19419993413,5cyclictest23302-21/usr/sbin/munin06:10:513
1941999341,19cyclictest17856-21latency_hist08:10:423
1941799341,27cyclictest12786-21uniq05:40:581
1941999338,17cyclictest0-21swapper/308:25:213
1941999334,15cyclictest0-21swapper/308:35:513
1941999333,5cyclictest7850irq/56-dwc_otg_07:06:073
1941999331,4cyclictest23520-21ssh07:00:413
1941999331,3cyclictest7950irq/56-dwc_otg_08:41:063
1941999331,24cyclictest0-21swapper/306:50:423
1941999331,13cyclictest0-21swapper/309:13:223
1941799333,14cyclictest7950irq/56-dwc_otg_05:50:581
1941799332,2cyclictest0-21swapper/109:46:001
1941799332,21cyclictest0-21swapper/106:22:411
1941799332,15cyclictest5733-21/usr/sbin/munin07:56:091
1941799331,5cyclictest7750irq/56-dwc_otg04:50:571
19417993314,5cyclictest23-21ksoftirqd/106:56:001
1941799331,15cyclictest23629-21tune2fs06:10:571
1941999328,6cyclictest7850irq/56-dwc_otg_06:41:103
1941999327,13cyclictest7950irq/56-dwc_otg_04:35:413
1941999327,13cyclictest7950irq/56-dwc_otg_04:35:413
1941999326,4cyclictest7850irq/56-dwc_otg_09:35:423
1941999323,19cyclictest21842-21sort06:05:583
1941999321,7cyclictest25577-21systemd-journal07:45:423
1941799328,4cyclictest7850irq/56-dwc_otg_09:35:591
1941799328,4cyclictest7850irq/56-dwc_otg_09:35:591
1941799327,13cyclictest7950irq/56-dwc_otg_08:00:561
1941799325,12cyclictest7950irq/56-dwc_otg_07:06:101
1941799322,13cyclictest7750irq/56-dwc_otg07:25:591
19417993215,5cyclictest1441550wget05:45:551
1941999317,14cyclictest0-21swapper/307:25:543
1941999313,5cyclictest7950irq/56-dwc_otg_06:21:093
1941999311,20cyclictest11659-21timerwakeupswit05:36:083
1941999311,20cyclictest11659-21timerwakeupswit05:36:083
1941799313,5cyclictest7750irq/56-dwc_otg06:05:411
19417993122,6cyclictest23-21ksoftirqd/105:36:041
19417993122,6cyclictest23-21ksoftirqd/105:36:041
1941999307,5cyclictest7850irq/56-dwc_otg_07:50:533
1941999306,13cyclictest7850irq/56-dwc_otg_05:11:073
1941999306,13cyclictest7850irq/56-dwc_otg_05:11:073
1941999305,5cyclictest7950irq/56-dwc_otg_05:55:573
1941999305,12cyclictest7950irq/56-dwc_otg_08:16:023
1941999305,10cyclictest7950irq/56-dwc_otg_04:53:543
1941999304,6cyclictest7950irq/56-dwc_otg_06:15:443
1941999304,13cyclictest7950irq/56-dwc_otg_04:41:023
1941999303,6cyclictest7950irq/56-dwc_otg_06:26:073
19419993024,4cyclictest12612-21idleruntime-cro07:25:413
1941999302,15cyclictest2491-21ssh06:35:533
1941999301,6cyclictest7850irq/56-dwc_otg_09:20:593
19419992922,4cyclictest30300-21aten_rbpower_cu05:00:553
1941999291,19cyclictest0-21swapper/309:42:573
1941799299,5cyclictest23789-21/usr/sbin/munin04:41:041
1941999284,5cyclictest25737-21/usr/sbin/munin07:41:043
1941999281,24cyclictest535-21timerandwakeup05:06:083
1941999266,5cyclictest7950irq/56-dwc_otg_05:51:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional