You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 04:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Apr 12, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3067621770,5sleep12089499cyclictest09:41:271
67572950,6sleep22089599cyclictest05:20:522
183062800,5sleep02089399cyclictest09:26:360
183062800,5sleep02089399cyclictest09:26:360
317572720,2sleep10-21swapper/107:46:041
317572720,2sleep10-21swapper/107:46:041
2089399711,53cyclictest0-21swapper/009:40:520
2089399709,35cyclictest0-21swapper/009:55:550
2089699696,35cyclictest7850irq/56-dwc_otg_08:40:543
2089699696,34cyclictest7950irq/56-dwc_otg_08:35:533
52062670,2sleep10-21swapper/107:15:241
20893996542,6cyclictest7750irq/56-dwc_otg09:46:110
20893996339,14cyclictest0-21swapper/008:11:020
20893996239,4cyclictest15870-21aten_rbpower_po05:45:570
20893996228,21cyclictest7750irq/56-dwc_otg05:00:420
20893996228,21cyclictest7750irq/56-dwc_otg05:00:420
20893996228,17cyclictest0-21swapper/005:55:540
2089399621,47cyclictest15570-21sed07:26:110
2089399621,45cyclictest28188-21cut08:21:020
20893996148,5cyclictest7750irq/56-dwc_otg05:11:100
20893996133,7cyclictest7750irq/56-dwc_otg08:51:060
20893996128,15cyclictest7850irq/56-dwc_otg_08:31:040
2089399612,50cyclictest0-21swapper/008:10:420
2089399611,41cyclictest0-21swapper/007:45:550
2089399611,41cyclictest0-21swapper/007:45:550
2089499608,16cyclictest7750irq/56-dwc_otg08:50:541
20893996042,3cyclictest0-21swapper/009:21:080
2089399602,42cyclictest11300-21ssh07:21:100
2089399601,42cyclictest25707-21ssh07:00:590
2089399599,29cyclictest0-21swapper/007:35:560
2089399599,26cyclictest7154-21fschecks_time07:56:000
2089399592,8cyclictest9335-21grep09:16:080
2089399592,44cyclictest0-21swapper/008:00:590
2089399592,44cyclictest0-21swapper/004:42:000
20894995837,9cyclictest0-21swapper/107:15:531
20894995837,9cyclictest0-21swapper/107:15:531
20893995842,3cyclictest0-21swapper/009:31:090
20893995831,16cyclictest0-21swapper/008:46:060
20893995831,10cyclictest2991-21uniq07:51:010
20893995812,22cyclictest32328-21uname08:26:030
2089699572,8cyclictest7950irq/56-dwc_otg_04:35:513
2089399574,49cyclictest0-21swapper/005:06:100
2089399574,23cyclictest0-21swapper/008:40:550
20893995738,6cyclictest26359-21grep09:36:100
2089399571,47cyclictest908-21runrttasks07:34:490
2089399571,47cyclictest0-21swapper/008:17:080
2089399571,42cyclictest0-21swapper/004:34:190
74502560,5sleep02089399cyclictest09:14:280
2089399563,31cyclictest0-21swapper/008:56:060
2089399563,31cyclictest0-21swapper/008:56:060
2089399562,40cyclictest0-21swapper/009:51:130
20893995623,17cyclictest5602-21ps05:16:070
2089499558,9cyclictest0-21swapper/108:40:531
2089399551,35cyclictest28807-21fschecks_time09:01:000
20893995438,3cyclictest0-21swapper/009:06:080
20893995432,4cyclictest7850irq/56-dwc_otg_05:55:420
2089399542,42cyclictest0-21swapper/008:39:240
2089399542,39cyclictest23067-21cut04:35:580
2089599533,32cyclictest7950irq/56-dwc_otg_08:01:072
20893995331,13cyclictest208542sleep006:55:420
2089699526,17cyclictest0-21swapper/305:45:563
2089699523,4cyclictest7950irq/56-dwc_otg_06:30:553
2089399523,7cyclictest0-21swapper/007:40:540
20893995230,3cyclictest0-21swapper/006:45:570
2089399522,27cyclictest8547-21/usr/sbin/munin05:26:010
2089399521,34cyclictest0-21swapper/006:06:120
2089399521,31cyclictest14218-21fschecks_count05:40:570
2089399521,31cyclictest12282-21grep05:35:570
20894995124,8cyclictest7950irq/56-dwc_otg_04:50:571
2089399519,22cyclictest0-21swapper/006:55:590
2089399511,27cyclictest1645-21/usr/sbin/munin07:10:570
2089399511,25cyclictest7424-21ntp_kernel_pll_05:21:030
2089499504,10cyclictest29663-21runrttasks07:05:561
2089499504,10cyclictest29663-21runrttasks07:05:561
20893995028,9cyclictest0-21swapper/007:15:530
20893995028,9cyclictest0-21swapper/007:15:530
20893995026,5cyclictest10635-21fschecks_count05:31:000
2089399492,37cyclictest29919-21latency_hist04:55:430
2089399491,34cyclictest0-21swapper/006:42:190
111212490,5sleep225577-21systemd-journal09:18:372
2089499485,13cyclictest7750irq/56-dwc_otg06:40:541
20894994824,5cyclictest6476-21cron07:55:411
20893994824,13cyclictest908-21runrttasks06:30:490
2089399481,3cyclictest0-21swapper/006:30:430
2089399481,33cyclictest0-21swapper/006:21:500
202932480,2sleep20-21swapper/206:54:562
2089699477,26cyclictest0-21swapper/304:40:543
2089599477,23cyclictest0-21swapper/205:00:542
2089499478,25cyclictest0-21swapper/106:15:561
20894994713,9cyclictest7850irq/56-dwc_otg_08:35:551
2089399477,28cyclictest0-21swapper/005:00:540
2089399471,35cyclictest0-21swapper/006:36:060
2089499467,24cyclictest0-21swapper/105:40:531
20894994620,4cyclictest7750irq/56-dwc_otg08:21:091
20894994617,12cyclictest7750irq/56-dwc_otg09:30:561
2089399463,30cyclictest29629-21ssh07:05:530
2089399463,30cyclictest29629-21ssh07:05:530
20893994622,13cyclictest0-21swapper/006:10:560
2089399461,35cyclictest0-21swapper/006:01:250
2089399461,35cyclictest0-21swapper/006:01:250
239522450,4sleep223951-21if_eth008:16:002
2089699456,25cyclictest0-21swapper/304:45:563
2089599452,29cyclictest7950irq/56-dwc_otg_07:36:002
2089499457,24cyclictest0-21swapper/108:30:551
2089499454,6cyclictest7750irq/56-dwc_otg04:30:521
20894994522,4cyclictest7750irq/56-dwc_otg05:21:081
2089399452,31cyclictest0-21swapper/004:47:290
2089399451,36cyclictest0-21swapper/006:17:280
205532453,36sleep00-21swapper/004:29:470
12682450,4sleep32089699cyclictest07:10:433
12682450,4sleep32089699cyclictest07:10:433
2089699446,24cyclictest32645-21ssh09:05:553
2089499447,24cyclictest0-21swapper/105:30:561
2089499447,24cyclictest0-21swapper/105:15:561
2089499441,8cyclictest7750irq/56-dwc_otg06:45:551
20894994410,12cyclictest12035-21sh09:58:441
238142430,6sleep32089699cyclictest08:55:233
2089699437,22cyclictest0-21swapper/304:30:553
2089699437,21cyclictest0-21swapper/306:15:563
2089699431,32cyclictest0-21swapper/307:55:423
2089499435,18cyclictest0-21swapper/107:31:381
20894994317,8cyclictest7950irq/56-dwc_otg_07:42:491
20894994312,15cyclictest7950irq/56-dwc_otg_09:00:431
20894994312,15cyclictest7950irq/56-dwc_otg_09:00:431
87392420,2sleep2301ktimersoftd/209:55:292
2089499428,18cyclictest7850irq/56-dwc_otg_09:22:591
2089499426,11cyclictest7950irq/56-dwc_otg_06:10:411
20894994221,4cyclictest8878-21idleruntime-cro09:55:421
20894994214,9cyclictest19147-21taskset09:28:081
20894994214,9cyclictest19147-21taskset09:28:081
2089699417,21cyclictest0-21swapper/306:25:563
2089699413,8cyclictest7950irq/56-dwc_otg_08:45:533
2089599412,8cyclictest0-21swapper/204:30:532
2089499418,19cyclictest28227-21ssh09:39:321
2089499416,13cyclictest221ktimersoftd/107:25:591
20894994119,8cyclictest7750irq/56-dwc_otg06:10:531
20894994119,3cyclictest7750irq/56-dwc_otg05:51:021
20894994110,6cyclictest221ktimersoftd/108:27:151
2089699407,24cyclictest0-21swapper/305:20:543
2089699407,21cyclictest0-21swapper/306:40:563
2089699402,10cyclictest7950irq/56-dwc_otg_08:25:543
20894994019,10cyclictest7750irq/56-dwc_otg04:56:031
20894994019,10cyclictest7750irq/56-dwc_otg04:56:031
2089499401,12cyclictest0-21swapper/108:18:371
2089699396,19cyclictest0-21swapper/305:35:563
20896993921,4cyclictest12654-21aten_rbpower_po06:45:573
2089699391,4cyclictest7950irq/56-dwc_otg_09:45:543
2089599397,5cyclictest24162-21unixbench_singl06:06:122
2089499397,15cyclictest18759-21chrt08:10:161
20894993910,10cyclictest4063-21cut06:35:571
198662390,3sleep10-21swapper/108:50:421
2089699389,13cyclictest381ktimersoftd/307:50:433
2089699389,13cyclictest381ktimersoftd/307:50:433
2089699384,4cyclictest7950irq/56-dwc_otg_09:55:553
2089699384,16cyclictest37-21rcuc/304:55:593
2089699384,16cyclictest37-21rcuc/304:55:593
20895993817,5cyclictest22702-21sh09:32:022
20894993815,6cyclictest20556-21ssh06:55:241
20894993811,6cyclictest23-21ksoftirqd/105:45:581
2089499381,16cyclictest0-21swapper/107:39:391
2089699377,23cyclictest0-21swapper/305:40:533
2089699371,6cyclictest21083-21ssh06:55:533
2089699371,23cyclictest20408-21grep05:56:103
20896993711,5cyclictest0-21swapper/309:21:023
2089599375,15cyclictest7950irq/56-dwc_otg_09:40:422
20895993715,7cyclictest3088-21aten_rbpower_en05:10:562
20895993714,8cyclictest0-21swapper/207:26:122
2089599371,25cyclictest26443-21ls06:15:422
2089499377,13cyclictest7950irq/56-dwc_otg_07:00:581
2089499372,13cyclictest32722-21ssh06:31:031
20894993716,8cyclictest7850irq/56-dwc_otg_07:23:151
20894993715,7cyclictest7950irq/56-dwc_otg_09:11:511
20894993715,6cyclictest8778-21sh07:57:201
20894993713,12cyclictest7750irq/56-dwc_otg07:00:351
2089499371,21cyclictest21694-21ssh08:13:011
2089499371,21cyclictest0-21swapper/105:35:561
20894993711,9cyclictest19746-21idleruntime05:56:001
116262371,2sleep10-21swapper/109:19:341
2089699366,16cyclictest0-21swapper/307:26:053
2089699361,24cyclictest6892-21timerandwakeup07:16:093
2089699361,24cyclictest6892-21timerandwakeup07:16:093
20896993610,7cyclictest14883-21latency_hist08:05:423
20896993610,5cyclictest0-21swapper/309:00:433
20896993610,5cyclictest0-21swapper/309:00:433
20895993619,10cyclictest8250-21/usr/sbin/munin06:41:082
20895993616,8cyclictest7950irq/56-dwc_otg_07:22:372
2089599361,20cyclictest4138-21ssh09:10:422
20894993625,7cyclictest221ktimersoftd/104:40:531
2089499362,16cyclictest7950irq/56-dwc_otg_09:00:431
20894993619,8cyclictest7750irq/56-dwc_otg08:05:001
20894993611,13cyclictest0-21swapper/105:00:481
20894993610,7cyclictest0-21swapper/104:45:541
199032360,2sleep20-21swapper/204:26:072
65412350,5sleep3381ktimersoftd/308:34:213
2089699357,19cyclictest0-21swapper/306:50:563
2089699351,25cyclictest15118-21ssh08:05:473
20896993512,4cyclictest0-21swapper/307:55:573
2089599357,7cyclictest7850irq/56-dwc_otg_05:41:002
20895993514,6cyclictest2113850wget06:55:562
2089599351,24cyclictest13007-21ls09:21:012
2089599351,15cyclictest7750irq/56-dwc_otg07:31:002
2089499358,15cyclictest0-21swapper/109:05:551
2089499356,4cyclictest1452-21cut09:46:031
203662352,25sleep10-21swapper/104:27:211
78922340,5sleep32089699cyclictest06:40:393
295012340,4sleep229502-21unixbench_multi06:21:102
2089699346,6cyclictest7750irq/56-dwc_otg07:20:593
2089699346,15cyclictest0-21swapper/306:20:563
2089699346,13cyclictest7950irq/56-dwc_otg_09:30:523
2089699342,5cyclictest7950irq/56-dwc_otg_07:00:543
20896993413,7cyclictest7950irq/56-dwc_otg_05:15:423
2089699341,24cyclictest10202-21ls05:30:423
20896993412,11cyclictest7850irq/56-dwc_otg_05:07:263
2089699341,16cyclictest0-21swapper/306:10:543
2089699341,13cyclictest39-21ksoftirqd/309:53:263
2089599349,5cyclictest0-21swapper/209:55:572
2089599346,22cyclictest25577-21systemd-journal07:47:072
2089599346,22cyclictest25577-21systemd-journal07:47:072
2089599346,17cyclictest0-21swapper/209:27:572
2089599346,17cyclictest0-21swapper/209:27:572
2089599345,4cyclictest7750irq/56-dwc_otg04:41:082
2089599342,6cyclictest12732-21ssh08:41:082
20895993418,5cyclictest15717-21sh08:06:022
2089599341,7cyclictest0-21swapper/208:20:532
2089599341,22cyclictest8266-21ls08:36:032
2089599341,14cyclictest7950irq/56-dwc_otg_05:46:002
2089499343,6cyclictest22326-21unixbench_multi06:01:101
2089499343,6cyclictest22326-21unixbench_multi06:01:101
2089699339,7cyclictest7750irq/56-dwc_otg08:16:093
2089699337,15cyclictest7950irq/56-dwc_otg_05:51:103
2089699336,6cyclictest7950irq/56-dwc_otg_04:50:553
2089699336,15cyclictest39-21ksoftirqd/308:10:563
2089699332,14cyclictest0-21swapper/309:41:243
20896993315,8cyclictest27115-21fschecks_count07:40:593
2089599337,19cyclictest0-21swapper/206:46:442
2089599336,6cyclictest7850irq/56-dwc_otg_07:00:592
2089599332,14cyclictest7750irq/56-dwc_otg07:51:052
20895993317,5cyclictest20162-21/usr/sbin/munin08:51:052
20895993314,10cyclictest19771-21sh08:50:352
20895993313,4cyclictest31796-21diskmemload06:37:492
2089599331,22cyclictest32008-21cpuspeed08:25:572
2089499338,11cyclictest23-21ksoftirqd/105:10:061
2089499332,3cyclictest7750irq/56-dwc_otg06:20:541
2089699325,17cyclictest908-21runrttasks07:36:143
20896993224,6cyclictest37-21rcuc/306:01:093
20896993224,6cyclictest37-21rcuc/306:01:093
2089699322,22cyclictest0-21swapper/307:35:413
2089699321,20cyclictest4643-21cut09:10:593
20896993211,12cyclictest39-21ksoftirqd/307:11:393
2089599329,4cyclictest0-21swapper/207:11:042
2089599328,5cyclictest7850irq/56-dwc_otg_08:31:082
2089599328,11cyclictest7850irq/56-dwc_otg_05:16:112
2089599327,4cyclictest7850irq/56-dwc_otg_08:58:052
2089599327,4cyclictest7850irq/56-dwc_otg_08:58:052
2089599327,4cyclictest7850irq/56-dwc_otg_07:56:022
2089599327,15cyclictest0-21swapper/207:15:562
2089599327,15cyclictest0-21swapper/207:15:562
2089599326,5cyclictest7850irq/56-dwc_otg_06:00:592
2089599326,5cyclictest7850irq/56-dwc_otg_06:00:592
2089599325,6cyclictest7850irq/56-dwc_otg_05:36:022
20895993215,5cyclictest2177-21/usr/sbin/munin09:46:112
2089599321,22cyclictest28482-21uniq04:50:582
20894993226,4cyclictest221ktimersoftd/105:10:551
2089699318,20cyclictest0-21swapper/305:30:563
2089699317,9cyclictest39-21ksoftirqd/308:24:073
2089699317,14cyclictest0-21swapper/305:15:563
2089699316,14cyclictest0-21swapper/305:00:483
2089699311,6cyclictest1849150ssh09:26:563
2089699311,6cyclictest1849150ssh09:26:563
20896993116,9cyclictest11422-21ssh09:19:103
2089599319,10cyclictest0-21swapper/209:40:572
2089599317,5cyclictest7950irq/56-dwc_otg_09:11:112
2089599317,11cyclictest30547-21cut06:26:002
2089599315,7cyclictest7850irq/56-dwc_otg_09:01:102
2089599311,20cyclictest1144-21ssh07:10:422
2089599311,20cyclictest1144-21ssh07:10:422
20895993111,4cyclictest27135-21fschecks_count07:40:592
20895993110,11cyclictest22262-1kworker/u9:108:14:292
2089499316,6cyclictest7750irq/56-dwc_otg06:26:051
2089499316,15cyclictest0-21swapper/105:25:551
2089699308,4cyclictest0-21swapper/309:01:163
2089699301,20cyclictest0-21swapper/309:35:513
2089599306,7cyclictest7950irq/56-dwc_otg_04:45:562
2089599303,12cyclictest7950irq/56-dwc_otg_05:31:022
2089599302,11cyclictest29943-21/usr/sbin/munin04:56:082
2089599302,11cyclictest29943-21/usr/sbin/munin04:56:082
2089599292,6cyclictest7850irq/56-dwc_otg_06:31:112
2089599291,14cyclictest7750irq/56-dwc_otg05:26:052
2089599291,12cyclictest7850irq/56-dwc_otg_05:55:572
2089499291,8cyclictest23301-21ls04:36:011
20896992811,8cyclictest39-21ksoftirqd/306:10:343
2089599284,6cyclictest7850irq/56-dwc_otg_05:50:582
2089599282,15cyclictest0-21swapper/206:17:192
205712284,15sleep30-21swapper/304:30:023
2089599277,15cyclictest0-21swapper/205:05:552
2089599264,4cyclictest7750irq/56-dwc_otg04:36:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional