You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 04:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Apr 07, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
902121580,5sleep21084599cyclictest06:53:192
2211421210,5sleep01084099cyclictest07:09:310
1883821060,6sleep11084499cyclictest07:44:391
123352990,2sleep00-21swapper/006:56:470
95622900,1sleep20-21swapper/204:26:002
1084099831,56cyclictest0-21swapper/005:55:520
10840998263,5cyclictest7850irq/56-dwc_otg_06:46:080
213242790,2sleep30-21swapper/307:08:053
10840997917,15cyclictest0-21swapper/009:50:520
288382770,6sleep11084499cyclictest06:38:201
277152760,5sleep21084599cyclictest07:55:392
1084499767,32cyclictest0-21swapper/107:20:531
1084099731,58cyclictest20171-21munin-plugin-st08:25:400
1084599712,19cyclictest7850irq/56-dwc_otg_05:10:522
1084099684,42cyclictest0-21swapper/005:15:490
10840996720,5cyclictest18022-21latency_hist06:20:390
10844996627,18cyclictest7850irq/56-dwc_otg_04:55:511
10840996643,14cyclictest115582chrt07:35:520
268512650,1sleep20-21swapper/207:15:372
10840996559,3cyclictest0-21swapper/008:55:530
10840996522,8cyclictest0-21swapper/008:35:510
1084099651,41cyclictest0-21swapper/009:25:520
1084099642,22cyclictest0-21swapper/008:30:500
10846996310,10cyclictest7850irq/56-dwc_otg_08:35:513
1084099634,11cyclictest0-21swapper/005:35:490
1084099631,52cyclictest10005-21ssh08:51:080
1084499624,14cyclictest7750irq/56-dwc_otg07:55:511
10840996223,24cyclictest0-21swapper/004:35:590
10840995926,5cyclictest1-21systemd04:55:510
1084099592,39cyclictest0-21swapper/007:50:570
1084099586,39cyclictest0-21swapper/006:36:010
1084099581,32cyclictest0-21swapper/007:25:120
1084099579,18cyclictest0-21swapper/008:05:500
1084099574,17cyclictest0-21swapper/008:10:520
1084099572,33cyclictest0-21swapper/008:40:510
10840995624,17cyclictest7850irq/56-dwc_otg_07:15:390
1084099562,30cyclictest0-21swapper/005:26:070
1084099561,6cyclictest21637-21diskmemload06:51:070
10840995616,34cyclictest908-21runrttasks07:01:040
124062550,25sleep27950irq/56-dwc_otg_09:33:592
124062550,25sleep27950irq/56-dwc_otg_09:33:592
1084099557,31cyclictest0-21swapper/009:25:400
10840995428,5cyclictest7850irq/56-dwc_otg_09:56:030
1084099542,32cyclictest23246-21grep06:31:070
1084099541,30cyclictest18408-21grep09:01:070
1084099541,30cyclictest18408-21grep09:01:070
1084099541,24cyclictest13424-21grep08:16:070
1084099531,6cyclictest28518-21ssh07:55:590
1084099531,4cyclictest19978-21ssh07:45:560
10840995313,25cyclictest15650-21/usr/sbin/munin07:40:540
1084099531,25cyclictest22143-21ssh09:45:540
10845995210,11cyclictest7950irq/56-dwc_otg_08:25:522
1084499521,15cyclictest19721-21ssh07:45:501
1084099526,27cyclictest0-21swapper/006:25:400
1084099523,41cyclictest31961-21kworker/0:209:31:070
1084099523,41cyclictest31961-21kworker/0:209:31:070
1084099522,6cyclictest12970-21aten_rbpower_vo06:05:540
10840995225,15cyclictest0-21swapper/006:10:520
10840995216,18cyclictest8586-21unixbench_singl07:31:080
10840995210,21cyclictest0-21swapper/007:19:530
212432510,25sleep17950irq/56-dwc_otg_08:26:021
1084099515,25cyclictest30374-21ssh09:16:040
10840995130,5cyclictest7950irq/56-dwc_otg_08:45:570
10840995130,5cyclictest7950irq/56-dwc_otg_08:45:570
1084099512,28cyclictest13712-21aten_rbpower_cu09:35:530
1084099511,30cyclictest25103-21munin-plugin-st09:10:390
315062500,2sleep30-21swapper/309:56:063
1084099509,28cyclictest0-21swapper/008:30:390
10840995025,13cyclictest7950irq/56-dwc_otg_06:41:070
10840995023,10cyclictest7950irq/56-dwc_otg_04:45:390
1084099502,29cyclictest0-21swapper/005:45:390
1084099502,29cyclictest0-21swapper/005:45:390
1084099501,30cyclictest25584-21fschecks_count05:10:560
10845994914,10cyclictest7850irq/56-dwc_otg_07:35:492
1084099492,44cyclictest0-21swapper/009:11:000
1084099492,29cyclictest0-21swapper/006:01:000
1084099491,4cyclictest0-21swapper/009:40:530
1084699487,26cyclictest0-21swapper/304:50:523
1084499482,10cyclictest23086-21ssh07:10:521
1084499481,3cyclictest0-21swapper/106:30:511
1084099486,28cyclictest18641-21ls04:50:590
1084099486,27cyclictest101ktimersoftd/005:31:000
10840994826,5cyclictest7950irq/56-dwc_otg_06:25:560
10840994818,17cyclictest11391-21grep04:30:580
1084099481,4cyclictest32119-21ssh08:00:490
10846994715,15cyclictest0-21swapper/307:00:573
10844994721,24cyclictest0-21swapper/104:50:521
10840994727,9cyclictest7950irq/56-dwc_otg_05:06:030
1084699467,24cyclictest0-21swapper/307:50:533
10846994620,4cyclictest7540-21aten_rbpower_vo05:50:523
10840994615,4cyclictest7415-21/usr/sbin/munin05:51:050
10840994614,15cyclictest0-21swapper/005:50:390
104542463,35sleep00-21swapper/004:29:060
70442450,31sleep30-21swapper/306:50:583
116422450,2sleep20-21swapper/208:53:572
1084499457,24cyclictest0-21swapper/106:15:521
1084499457,24cyclictest0-21swapper/105:15:521
10844994513,6cyclictest7950irq/56-dwc_otg_09:45:521
1084099457,18cyclictest101ktimersoftd/005:00:580
1084099457,18cyclictest101ktimersoftd/005:00:580
1084099452,21cyclictest0-21swapper/005:21:010
1084699447,25cyclictest0-21swapper/306:25:543
1084699444,6cyclictest0-21swapper/309:25:523
1084599447,6cyclictest31-21ksoftirqd/206:06:042
1084599447,22cyclictest0-21swapper/205:45:522
10845994411,7cyclictest7750irq/56-dwc_otg10:00:392
1084499445,24cyclictest30259-21ssh09:55:451
10844994435,7cyclictest221ktimersoftd/107:15:511
1084499443,19cyclictest0-21swapper/108:05:501
1084699437,24cyclictest0-21swapper/309:00:453
1084699437,24cyclictest0-21swapper/309:00:453
1084599435,4cyclictest0-21swapper/208:40:502
10844994323,4cyclictest7850irq/56-dwc_otg_09:25:521
10840994329,4cyclictest7950irq/56-dwc_otg_07:30:390
285382420,4sleep311-21rcu_preempt09:14:233
1084599427,22cyclictest24646-21ssh08:30:522
1084499427,25cyclictest0-21swapper/106:50:521
1084499422,10cyclictest0-21swapper/108:45:521
1084499422,10cyclictest0-21swapper/108:45:521
10844994216,6cyclictest23-21ksoftirqd/107:00:581
1084099421,7cyclictest16567-21fschecks_count04:45:560
102252425,26sleep39024-21/usr/sbin/munin04:26:103
1084699411,25cyclictest0-21swapper/305:00:523
1084699411,25cyclictest0-21swapper/305:00:523
1084499414,23cyclictest0-21swapper/106:20:541
1084499413,9cyclictest7950irq/56-dwc_otg_09:50:551
10844994122,10cyclictest7299-21sh07:30:521
10844994110,15cyclictest7950irq/56-dwc_otg_09:01:021
10844994110,15cyclictest7950irq/56-dwc_otg_09:01:021
1084699404,18cyclictest7850irq/56-dwc_otg_08:06:073
1084699401,24cyclictest0-21swapper/304:40:523
1084599401,5cyclictest0-21swapper/207:20:522
10845994011,15cyclictest10832-21cyclictest08:13:162
10844994020,11cyclictest11943-21/usr/sbin/munin04:31:051
276002390,2sleep229-21rcuc/207:15:572
113372390,1sleep10-21swapper/107:35:411
109692391,5sleep310978-21diskmemload09:31:233
109692391,5sleep310978-21diskmemload09:31:233
10844993918,5cyclictest14460-21sh09:36:031
1084599383,9cyclictest19742-21ssh07:45:522
1084599382,27cyclictest0-21swapper/208:10:392
10845993818,5cyclictest7950irq/56-dwc_otg_08:20:502
303572370,1sleep20-21swapper/209:16:032
1084699372,20cyclictest7950irq/56-dwc_otg_06:45:393
10846993712,5cyclictest18574-21ntpq09:41:013
1084599377,25cyclictest0-21swapper/205:05:522
1084599376,4cyclictest7950irq/56-dwc_otg_08:35:572
1084599375,5cyclictest7850irq/56-dwc_otg_09:46:052
1084599374,6cyclictest7850irq/56-dwc_otg_08:50:392
1084599374,6cyclictest7850irq/56-dwc_otg_08:50:392
1084599371,9cyclictest0-21swapper/204:50:542
10845993713,5cyclictest18363-21latency09:40:592
1084499377,25cyclictest0-21swapper/105:00:521
1084499377,25cyclictest0-21swapper/105:00:521
1084499376,12cyclictest7850irq/56-dwc_otg_09:21:011
10844993715,7cyclictest22324-21/usr/sbin/munin09:06:051
1084699365,19cyclictest8795-21ssh08:50:533
1084699365,10cyclictest7950irq/56-dwc_otg_07:35:523
1084699361,25cyclictest0-21swapper/308:10:533
1084499369,4cyclictest0-21swapper/108:36:071
1084499367,5cyclictest7750irq/56-dwc_otg09:20:391
10844993619,5cyclictest11191-21sh08:53:081
10846993525,4cyclictest7850irq/56-dwc_otg_08:27:223
1084699352,16cyclictest7950irq/56-dwc_otg_06:40:393
10846993515,10cyclictest22266-21fschecks_count09:45:563
10845993516,4cyclictest31-21ksoftirqd/206:36:152
1084599351,18cyclictest0-21swapper/207:03:452
1084499355,6cyclictest0-21swapper/105:35:491
1084499354,3cyclictest7850irq/56-dwc_otg_09:35:031
1084499354,3cyclictest7850irq/56-dwc_otg_09:35:031
1084499354,27cyclictest7950irq/56-dwc_otg_07:50:511
1084499353,7cyclictest7750irq/56-dwc_otg06:41:031
10844993522,4cyclictest13000-21wget08:55:541
1084499351,15cyclictest7750irq/56-dwc_otg04:46:061
1084699349,4cyclictest31933-21grep05:26:073
1084699346,6cyclictest7850irq/56-dwc_otg_06:00:493
1084699344,6cyclictest7950irq/56-dwc_otg_06:46:083
1084599345,16cyclictest31-21ksoftirqd/209:21:262
1084599344,14cyclictest7950irq/56-dwc_otg_09:25:562
1084599341,7cyclictest13829-21ssh08:56:052
1084599341,4cyclictest0-21swapper/207:25:542
1084499349,17cyclictest5022-21ssh07:27:201
1084499344,6cyclictest16397-21latency_hist06:15:401
10844993420,4cyclictest32697-21aten_rbpower_po05:30:521
1084499341,8cyclictest12762-21aten_rbpower_po04:35:531
1084499341,14cyclictest17152-21ssh08:21:031
205252330,3sleep20-21swapper/209:04:522
205252330,3sleep20-21swapper/209:04:522
1084699339,5cyclictest23215-21grep06:31:073
1084699338,6cyclictest27547-21fschecks_time07:15:573
10846993318,5cyclictest12884-21sh04:35:543
1084699331,6cyclictest0-21swapper/308:30:503
1084699331,22cyclictest0-21swapper/307:26:243
1084699331,21cyclictest29961-21ssh09:55:403
10846993310,16cyclictest0-21swapper/305:06:023
1084599337,6cyclictest7950irq/56-dwc_otg_04:40:562
1084599335,6cyclictest23306-21latency_hist05:05:392
1084599335,6cyclictest23306-21latency_hist05:05:392
10845993326,5cyclictest30022-21apt-config09:55:402
1084599332,5cyclictest11412-21ntp_states06:55:592
1084599332,19cyclictest0-21swapper/206:35:402
1084599331,6cyclictest8282-21ssh07:31:062
10845993314,4cyclictest29-21rcuc/204:36:392
10845993310,6cyclictest7750irq/56-dwc_otg05:25:442
10845993310,5cyclictest503-21in:imuxsock06:15:542
1084499339,5cyclictest6786-21grep05:46:061
1084499339,4cyclictest7750irq/56-dwc_otg09:42:241
1084499338,4cyclictest0-21swapper/108:10:521
1084499334,7cyclictest7750irq/56-dwc_otg07:06:021
1084499332,5cyclictest7750irq/56-dwc_otg08:41:001
1084499331,8cyclictest26706-21grep09:11:071
1084499331,18cyclictest14000-21grep06:06:081
1084499331,12cyclictest21616-21kworker/u8:208:30:561
1084699327,15cyclictest0-21swapper/307:30:453
1084699324,13cyclictest7950irq/56-dwc_otg_07:15:403
1084699321,5cyclictest7850irq/56-dwc_otg_08:01:063
1084599326,7cyclictest14350-1kworker/u9:206:45:532
1084599326,5cyclictest7850irq/56-dwc_otg_07:41:032
10845993215,7cyclictest7850irq/56-dwc_otg_07:07:032
1084599321,20cyclictest3770-21ls05:40:402
1084499327,22cyclictest0-21swapper/106:25:541
1084499324,7cyclictest11165-21aten_rbpower_vo06:00:521
1084499323,10cyclictest12785-21ssh08:15:591
10844993218,4cyclictest4039-21df_inode05:40:561
10844993218,4cyclictest4039-21df_inode05:40:561
1084499321,7cyclictest29922-21timerwakeupswit05:21:061
1084699318,7cyclictest908-21runrttasks07:25:123
1084699318,5cyclictest7950irq/56-dwc_otg_06:25:393
1084699317,13cyclictest7950irq/56-dwc_otg_08:40:533
10846993125,4cyclictest37-21rcuc/307:55:503
10846993120,4cyclictest2191-21switchtime09:21:053
1084699311,9cyclictest0-21swapper/306:12:173
10846993114,9cyclictest16080-1kworker/u9:108:59:493
10846993113,12cyclictest6458-21sh08:47:033
10846993113,12cyclictest6458-21sh08:47:033
10846993112,6cyclictest198482rm08:25:063
10846993110,5cyclictest0-21swapper/309:06:013
1084599314,5cyclictest3915-21aten_rbpower_po05:40:542
1084599314,5cyclictest3915-21aten_rbpower_po05:40:542
1084599314,14cyclictest7950irq/56-dwc_otg_07:55:572
1084599313,4cyclictest7850irq/56-dwc_otg_06:11:002
1084599313,4cyclictest7750irq/56-dwc_otg08:05:392
1084599313,19cyclictest11964-21grep04:31:052
1084599312,9cyclictest7850irq/56-dwc_otg_09:35:522
1084599312,9cyclictest0-21swapper/206:28:302
1084599312,17cyclictest0-21swapper/205:56:122
1084599311,8cyclictest21630-21sort09:05:562
10845993113,5cyclictest7850irq/56-dwc_otg_09:10:532
10845993111,10cyclictest29-21rcuc/205:22:472
1084499317,10cyclictest0-21swapper/106:56:461
1084499316,11cyclictest7950irq/56-dwc_otg_05:51:051
1084499315,18cyclictest7950irq/56-dwc_otg_05:55:521
1084499312,20cyclictest7750irq/56-dwc_otg06:45:521
1084699306,15cyclictest0-21swapper/304:30:513
1084699306,14cyclictest0-21swapper/305:40:543
1084699306,14cyclictest0-21swapper/305:40:543
1084699304,4cyclictest0-21swapper/309:15:593
1084699301,7cyclictest10902-21grep06:55:533
10846993011,12cyclictest0-21swapper/307:46:023
1084599309,8cyclictest29-21rcuc/206:22:482
1084599308,4cyclictest7950irq/56-dwc_otg_08:18:302
1084599304,5cyclictest31-21ksoftirqd/205:30:552
1084599304,4cyclictest7750irq/56-dwc_otg06:00:552
1084599303,15cyclictest27114-21/usr/sbin/munin05:15:572
10845993024,4cyclictest301ktimersoftd/206:40:542
1084599301,26cyclictest9177-21ls05:55:402
1084499303,5cyclictest7850irq/56-dwc_otg_05:26:021
1084499301,3cyclictest7950irq/56-dwc_otg_05:10:511
1084699296,16cyclictest0-21swapper/305:35:493
10846992923,4cyclictest12892-21aten_rbpower_en06:05:533
1084699291,8cyclictest12202-21ssh08:15:493
10846992912,9cyclictest286132chrt05:19:573
1084699291,16cyclictest0-21swapper/305:00:063
1084699291,13cyclictest7950irq/56-dwc_otg_05:31:063
1084699291,13cyclictest7950irq/56-dwc_otg_05:25:393
1084699291,12cyclictest7950irq/56-dwc_otg_07:40:543
10846992910,13cyclictest160732chrt09:38:063
1084599296,14cyclictest0-21swapper/204:55:532
1084599291,17cyclictest16538-21grep04:45:562
1084499297,5cyclictest7750irq/56-dwc_otg05:05:401
10844992910,6cyclictest32479-21fschecks_time08:00:551
1084699282,15cyclictest0-21swapper/305:46:053
1084699281,6cyclictest0-21swapper/305:10:423
1084699281,20cyclictest0-21swapper/304:47:033
1084699271,5cyclictest7950irq/56-dwc_otg_05:56:063
10846992710,5cyclictest17954-21sshd06:19:573
100192270,3sleep110025-21timerwakeupswit04:26:071
1084499261,15cyclictest0-21swapper/104:40:521
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional