You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 17:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun May 03, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3153221700,5sleep23199299cyclictest19:32:142
31980991156,71cyclictest0-21swapper/021:51:060
31980991056,70cyclictest0-21swapper/017:51:060
2176321040,27sleep03198099cyclictest17:31:110
31980991016,72cyclictest0-21swapper/017:06:070
225302950,5sleep13198499cyclictest20:40:531
3198099947,27cyclictest0-21swapper/016:41:060
3198099887,46cyclictest0-21swapper/018:51:070
77282840,2sleep30-21swapper/320:21:393
158902800,2sleep20-21swapper/217:16:042
3198099790,61cyclictest22495-21idleruntime-cro20:40:530
258032770,2sleep30-21swapper/318:47:473
3198099768,50cyclictest27671-21ls18:50:530
31980997656,5cyclictest32055-21apt-get16:30:590
89652740,2sleep10-21swapper/121:02:271
3198099746,50cyclictest0-21swapper/021:46:070
318342730,5sleep13198499cyclictest18:55:421
38712720,2sleep10-21swapper/119:00:531
31980997217,40cyclictest9-21ksoftirqd/020:16:070
31980997217,40cyclictest9-21ksoftirqd/020:16:070
3198099715,52cyclictest1055-21sed18:56:220
3198099711,28cyclictest0-21swapper/020:01:060
3199999701,6cyclictest7950irq/56-dwc_otg_18:26:073
3198099707,52cyclictest12422-21unixbench_multi21:06:220
31980996814,31cyclictest0-21swapper/019:06:170
3199299671,53cyclictest13415-21ssh20:29:292
3198099675,52cyclictest0-21swapper/021:31:120
3198099675,52cyclictest0-21swapper/021:31:120
31992996613,36cyclictest7950irq/56-dwc_otg_16:51:082
3199299651,51cyclictest32067-21munin-node16:31:032
3199299651,47cyclictest17432-21ssh19:55:302
3199299651,34cyclictest31730-21chrt17:58:472
3198099655,19cyclictest7843-21ssh21:41:060
3198099655,19cyclictest7843-21ssh21:41:060
3198099654,42cyclictest0-21swapper/021:26:070
3198099651,29cyclictest368150ssh21:36:070
31999996412,10cyclictest29316-21sed18:51:223
3199299641,48cyclictest22237-21ssh20:40:282
3198099649,35cyclictest17010-21ssh21:12:020
3198099648,46cyclictest32375-21ssh20:51:210
3198099642,31cyclictest3157-21latency_hist16:40:530
3199299631,51cyclictest25662-21ssh20:44:092
3199299631,51cyclictest22515-21ssh21:57:562
3199299631,51cyclictest22515-21ssh21:57:562
3199299631,43cyclictest6933-21ssh19:41:252
31980996310,35cyclictest17815-21wget17:21:070
3199999622,15cyclictest23163-21fschecks_time20:41:103
3199299621,43cyclictest6625-21ssh21:00:452
3199299621,41cyclictest9573-21awk18:26:182
31992996212,34cyclictest1411-21/usr/sbin/munin19:36:132
31980996216,24cyclictest10417-21diskmemload18:36:080
3198099621,45cyclictest30418-21/usr/sbin/munin17:56:070
3198099621,33cyclictest5155-21aten_rbpower_po16:46:080
3198099621,29cyclictest13533-21idleruntime-cro19:50:530
3198099621,29cyclictest13533-21idleruntime-cro19:50:530
3199999615,6cyclictest7950irq/56-dwc_otg_21:51:213
3199999611,6cyclictest7950irq/56-dwc_otg_17:21:213
3199299616,35cyclictest7950irq/56-dwc_otg_17:15:532
3199299616,35cyclictest7950irq/56-dwc_otg_17:15:532
3199299611,50cyclictest27321-21ssh19:27:182
3199299611,44cyclictest2342550ssh19:22:572
3198099619,37cyclictest23015-21run-parts18:45:220
31980996147,7cyclictest0-21swapper/021:20:260
3198099611,46cyclictest1690-21ssh19:36:090
3199999609,4cyclictest1350-21taskset19:35:573
3199299601,48cyclictest28667-21ssh18:51:142
3199299601,43cyclictest4344-21ssh20:18:022
3199299601,43cyclictest4344-21ssh20:18:022
3199299601,34cyclictest0-21swapper/221:51:312
3198099607,50cyclictest0-21swapper/018:06:070
3198099606,39cyclictest0-21swapper/019:55:330
3198099605,45cyclictest7244-21ls21:01:090
31980996036,5cyclictest26559-21grep20:06:100
3198099601,37cyclictest18202-21head19:16:220
3198099601,32cyclictest8023-21grep16:51:230
42442590,4sleep24242-21missed_timers21:36:142
3199299591,48cyclictest18430-21ssh18:39:482
3199299591,45cyclictest580-21ssh20:52:202
3199299591,44cyclictest32653-21ntp_states21:31:162
3199299591,44cyclictest32653-21ntp_states21:31:162
3199299591,40cyclictest16383-21ssh20:32:212
3198099597,36cyclictest7393-21ssh19:04:450
3198099596,43cyclictest3918-21cat20:56:200
3198099595,31cyclictest23873-21grep20:41:190
3198099592,39cyclictest11241-21rm20:26:180
3198099592,27cyclictest0-21swapper/019:41:210
3199299581,41cyclictest0-21swapper/221:26:042
3199299581,38cyclictest27782-21ssh20:06:442
3199299581,38cyclictest22170-21ssh20:01:072
3199299581,37cyclictest31964-21ssh20:11:482
3199299581,37cyclictest15018-21ssh19:13:092
3198099587,40cyclictest0-21swapper/020:35:350
3198099584,13cyclictest16578-21cron19:15:530
31980995814,41cyclictest101ktimersoftd/018:31:210
3198099580,42cyclictest6831-21latency_hist18:20:530
3199999575,9cyclictest7950irq/56-dwc_otg_17:56:153
3199999571,4cyclictest4898-21ntpq19:01:163
3199999571,15cyclictest0-21swapper/317:51:103
3199999571,14cyclictest0-21swapper/319:27:483
3199299572,41cyclictest0-21swapper/217:00:152
3199299571,43cyclictest29452-21chrt20:48:312
3199299571,43cyclictest17000-21ssh21:12:022
3199299571,42cyclictest10605-21ssh19:07:532
3199299571,41cyclictest9581-21ssh21:03:352
3199299571,29cyclictest0-21swapper/219:47:032
3199299571,29cyclictest0-21swapper/219:47:032
3198099577,33cyclictest0-21swapper/020:20:550
3198099571,32cyclictest24683-21sh17:40:530
3199999562,4cyclictest8393-21apt-get19:06:053
3199999561,9cyclictest0-21swapper/317:16:163
3199999561,6cyclictest2300250ssh20:01:193
3199299565,35cyclictest7950irq/56-dwc_otg_18:58:192
3199299561,43cyclictest1992-21sed16:36:142
3199299561,40cyclictest10163-21ssh21:43:132
3199299561,40cyclictest10163-21ssh21:43:132
3199299561,39cyclictest5217-21apt-get18:15:592
3199299561,34cyclictest7026-21/usr/sbin/munin20:21:162
3198099564,44cyclictest0-21swapper/018:12:430
3198099562,35cyclictest8598-21cron18:25:530
3198099562,35cyclictest8598-21cron18:25:530
3198099561,52cyclictest101ktimersoftd/017:16:150
3199999553,10cyclictest7950irq/56-dwc_otg_19:16:113
3199999552,14cyclictest0-21swapper/320:51:203
3199999551,9cyclictest0-21swapper/321:26:153
3199999551,18cyclictest0-21swapper/320:15:233
3199999551,11cyclictest0-21swapper/317:13:153
3199999551,11cyclictest0-21swapper/317:13:153
31992995515,37cyclictest22264-21sed17:31:182
3199299551,40cyclictest12720-21ssh21:06:452
3198099557,32cyclictest0-21swapper/021:56:170
3198099557,32cyclictest0-21swapper/021:56:170
3198099553,39cyclictest0-21swapper/019:27:400
3199999545,4cyclictest7750irq/56-dwc_otg20:30:413
3199999541,5cyclictest14599-21grep19:51:153
3199999541,15cyclictest0-21swapper/318:21:003
3199999541,15cyclictest0-21swapper/318:21:003
3199999541,13cyclictest0-21swapper/317:10:363
3199999541,12cyclictest0-21swapper/321:04:073
3199999541,11cyclictest0-21swapper/320:55:573
3199299541,42cyclictest20396-21ssh19:58:192
3199299541,40cyclictest0-21swapper/217:51:032
3199299541,39cyclictest13130-21ssh18:33:002
3199299541,36cyclictest20370-21/usr/sbin/munin21:16:202
3199299541,32cyclictest26218-21seq17:42:352
3198099548,32cyclictest0-21swapper/018:26:180
3198099545,38cyclictest0-21swapper/021:23:180
3198099543,41cyclictest0-21swapper/017:15:460
3198099543,41cyclictest0-21swapper/017:15:460
3199999538,4cyclictest0-21swapper/318:38:303
3199999532,13cyclictest12326-21fschecks_time21:46:113
3199999531,6cyclictest26777-21irqstats20:06:133
3199999531,18cyclictest0-21swapper/320:35:423
3199999531,12cyclictest7950irq/56-dwc_otg_16:33:523
3199299535,35cyclictest10417-21diskmemload19:03:222
3199299532,47cyclictest16933-21apt-get19:16:042
3199299531,36cyclictest24969-21ssh21:21:292
3199299531,34cyclictest7950irq/56-dwc_otg_16:45:532
3199299531,34cyclictest6518-21chrt16:49:392
3198099538,30cyclictest0-21swapper/019:22:000
3198099533,41cyclictest0-21swapper/020:49:250
3198099531,29cyclictest27404-21ntp_kernel_pll_17:46:180
31980995312,37cyclictest101ktimersoftd/019:35:470
3199999522,4cyclictest0-21swapper/318:42:593
3199999521,19cyclictest0-21swapper/320:49:393
3199299521,47cyclictest13351-21grep17:06:212
3199299521,40cyclictest23231-21ssh18:45:442
3199299521,37cyclictest7950irq/56-dwc_otg_17:50:542
31980995212,4cyclictest101ktimersoftd/017:26:210
3198099520,39cyclictest32314-21ssh20:12:240
3199999515,3cyclictest7750irq/56-dwc_otg21:14:033
31999995119,4cyclictest917-21snmpd17:29:383
31999995111,7cyclictest25442sleep321:34:383
31999995111,7cyclictest25442sleep321:34:383
3199999511,16cyclictest0-21swapper/318:12:293
3199999511,14cyclictest0-21swapper/320:18:113
3199999511,14cyclictest0-21swapper/320:18:113
3199999511,14cyclictest0-21swapper/319:58:503
3199299511,36cyclictest19500-21apt-get17:26:052
3198099516,30cyclictest17901-21sh19:55:550
3198099512,29cyclictest8762-21aten_rbpower_po16:56:060
3199999502,3cyclictest0-21swapper/319:21:173
3199999502,14cyclictest0-21swapper/318:04:423
3199999501,4cyclictest1445-21apt-get16:36:063
3199999501,15cyclictest0-21swapper/318:34:233
3199299501,46cyclictest0-21swapper/218:46:152
3199299501,38cyclictest13348-21ssh21:46:302
3199299501,32cyclictest0-21swapper/217:39:522
3199299501,28cyclictest7950irq/56-dwc_otg_17:21:132
31980995037,7cyclictest0-21swapper/017:02:320
3199999496,3cyclictest0-21swapper/319:46:333
3199999496,3cyclictest0-21swapper/319:46:333
3199999494,3cyclictest7750irq/56-dwc_otg18:56:223
3199999491,6cyclictest0-21swapper/321:24:053
3199999491,5cyclictest28282-21cron17:50:533
3199999491,5cyclictest108750ssh19:35:533
3199299493,35cyclictest909-21seq18:01:352
3199299491,44cyclictest11462-21ls17:01:192
3199999481,5cyclictest20600-21diskstats21:56:113
3199999481,5cyclictest20600-21diskstats21:56:113
3199299481,39cyclictest8387-21runrttasks18:23:172
3199299481,39cyclictest8387-21runrttasks18:23:172
3199299481,30cyclictest917-21snmpd18:14:472
31980994717,21cyclictest0-21swapper/018:03:260
3198099471,29cyclictest0-21swapper/017:40:540
3199999467,4cyclictest0-21swapper/319:11:223
316722465,35sleep00-21swapper/016:30:510
316722465,35sleep00-21swapper/016:30:510
3199299452,39cyclictest0-21swapper/216:30:542
3199299452,39cyclictest0-21swapper/216:30:542
3199299451,29cyclictest0-21swapper/218:07:162
31999994313,7cyclictest695050ssh19:41:273
3198499437,24cyclictest0-21swapper/120:50:591
3198499432,24cyclictest0-21swapper/119:11:171
31984994318,5cyclictest8704-21aten_rbpower_cu16:56:061
31984994314,22cyclictest10213-21ssh20:25:531
31984994219,6cyclictest13131-21switchtime17:06:191
3198499412,7cyclictest2152150ssh21:56:221
3198499412,7cyclictest2152150ssh21:56:221
3198499411,26cyclictest28768-21rm19:29:501
3199999401,32cyclictest0-21swapper/316:46:073
3198499402,25cyclictest14508-21/usr/sbin/munin20:31:111
31984994015,3cyclictest0-21swapper/120:06:101
3198499393,27cyclictest7950irq/56-dwc_otg_19:51:031
3198499391,24cyclictest3895-21missed_timers18:11:141
3198499386,5cyclictest7950irq/56-dwc_otg_18:36:081
31984993819,9cyclictest23639-21sh18:46:031
31984993818,5cyclictest8718-21wc18:25:541
31984993818,5cyclictest8718-21wc18:25:541
3198499371,6cyclictest7750irq/56-dwc_otg16:45:531
3199999361,22cyclictest7950irq/56-dwc_otg_21:36:103
3198499369,13cyclictest7950irq/56-dwc_otg_17:50:531
3198499361,5cyclictest0-21swapper/117:56:061
3198499361,27cyclictest23304-21ls17:36:081
31984993610,6cyclictest7850irq/56-dwc_otg_20:26:211
31984993610,13cyclictest0-21swapper/119:45:481
16862360,2sleep10-21swapper/120:15:451
3198499358,6cyclictest7950irq/56-dwc_otg_21:40:531
3198499357,6cyclictest7850irq/56-dwc_otg_18:41:071
3198499356,6cyclictest7750irq/56-dwc_otg21:45:531
3198499356,6cyclictest7750irq/56-dwc_otg21:45:531
31984993525,7cyclictest32215-21wget18:01:071
31984993517,4cyclictest12204-21sh21:46:091
31984993515,4cyclictest22155-21ssh20:01:061
3198499351,4cyclictest7750irq/56-dwc_otg21:06:061
3198499351,23cyclictest28706-21ls17:51:071
141032350,2sleep30-21swapper/321:09:183
3198499348,13cyclictest0-21swapper/120:56:451
3198499341,4cyclictest7750irq/56-dwc_otg16:55:531
3198499341,4cyclictest0-21swapper/119:38:521
3198499341,18cyclictest7750irq/56-dwc_otg17:21:071
31984993411,11cyclictest0-21swapper/121:53:381
31984993410,14cyclictest5407-21unixbench_singl19:01:221
307892346,14sleep30-21swapper/316:26:163
307892346,14sleep30-21swapper/316:26:163
3199999331,4cyclictest7950irq/56-dwc_otg_16:41:063
3198499338,7cyclictest10539-21ssh18:30:531
3198499338,5cyclictest0-21swapper/119:07:571
3198499337,23cyclictest0-21swapper/118:15:581
3198499332,9cyclictest0-21swapper/119:31:061
3198499332,15cyclictest7950irq/56-dwc_otg_17:05:541
31984993314,5cyclictest22992-21cpuspeed20:41:071
31984993311,7cyclictest6128-21unixbench-2d16:46:211
31984993310,5cyclictest908-21runrttasks17:16:081
31999993217,9cyclictest10417-21diskmemload20:39:453
3198499326,4cyclictest7850irq/56-dwc_otg_21:26:181
3198499326,12cyclictest7950irq/56-dwc_otg_19:46:091
3198499326,12cyclictest7950irq/56-dwc_otg_19:46:091
3198499321,6cyclictest7950irq/56-dwc_otg_18:31:071
3198499321,3cyclictest7850irq/56-dwc_otg_21:21:151
31984993210,5cyclictest7950irq/56-dwc_otg_20:50:531
311932323,16sleep131310-21/usr/sbin/munin16:26:221
311932323,16sleep131310-21/usr/sbin/munin16:26:221
3199999319,5cyclictest6350-21grep18:16:213
3199999314,15cyclictest7850irq/56-dwc_otg_16:51:233
3199999311,14cyclictest9811-21grep16:56:203
3198499318,10cyclictest0-21swapper/119:25:281
3198499313,7cyclictest0-21swapper/121:31:201
3198499313,7cyclictest0-21swapper/121:31:201
3198499312,4cyclictest20846-21ssh21:16:301
3198499311,6cyclictest19614-21grep17:26:081
3198499311,5cyclictest0-21swapper/119:16:121
31984993111,4cyclictest0-21swapper/120:16:091
31984993111,4cyclictest0-21swapper/120:16:091
3199999307,5cyclictest0-21swapper/321:41:033
3199999307,5cyclictest0-21swapper/321:41:033
31999993017,10cyclictest39-21ksoftirqd/317:01:043
3199999301,12cyclictest0-21swapper/321:15:553
31999993010,12cyclictest7950irq/56-dwc_otg_18:10:533
3198499306,7cyclictest7950irq/56-dwc_otg_21:11:091
3198499302,6cyclictest7950irq/56-dwc_otg_17:41:111
3198499301,22cyclictest14683-21missed_timers17:11:141
3198499301,22cyclictest14683-21missed_timers17:11:141
3199999297,4cyclictest7950irq/56-dwc_otg_17:36:163
3198499298,6cyclictest155550wget16:36:081
3199999281,17cyclictest0-21swapper/317:41:073
3199999281,14cyclictest21966-21tr17:31:143
3198499283,4cyclictest0-21swapper/119:56:051
3198499281,8cyclictest7850irq/56-dwc_otg_16:31:041
3198499278,12cyclictest25577-21systemd-journal17:34:391
3198499263,11cyclictest7850irq/56-dwc_otg_18:06:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional