You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 14:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Apr 14, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
238772790,6sleep01088199cyclictest07:50:210
10881997822,34cyclictest13-21rcuc/009:25:550
97232730,2sleep20-21swapper/207:32:112
10881997311,42cyclictest22102-21diskmemload07:10:560
319542710,2sleep031952-21sshd08:38:400
129892710,2sleep20-21swapper/206:57:002
10881996815,41cyclictest24787-21munin-plugin-st08:30:420
1088199681,52cyclictest1235-21cut08:41:000
1088199679,39cyclictest0-21swapper/007:36:120
1088199671,57cyclictest7337-21apt-get05:50:430
10881996713,37cyclictest17233-21fschecks_time08:21:010
323602650,2sleep2301ktimersoftd/209:56:492
29772650,5sleep21088399cyclictest07:25:102
10881996416,37cyclictest3076-21sh09:21:270
1088199640,49cyclictest4621-21kworker/0:009:06:060
10881996344,5cyclictest4056-21sort07:26:000
1088199626,43cyclictest12769-21aten_rbpower_en04:35:580
1088199621,53cyclictest0-21swapper/006:20:540
1088199612,44cyclictest0-21swapper/005:56:120
1088199602,45cyclictest854-21latency08:01:020
1088199602,43cyclictest22555-21aten_rbpower_po06:30:570
1088199602,36cyclictest0-21swapper/008:56:090
1088199601,48cyclictest7321-21ssh06:51:000
1088199601,41cyclictest0-21swapper/006:15:540
10881995942,8cyclictest7950irq/56-dwc_otg_07:51:060
1088299581,10cyclictest0-21swapper/107:10:561
1088199583,6cyclictest27760-21unixbench-2d09:51:120
1088199582,48cyclictest0-21swapper/005:30:590
10881995816,33cyclictest10381-21tune2fs09:31:000
1088199581,45cyclictest22797-21fschecks_time09:46:000
1088199581,41cyclictest30162-21grep05:21:110
1088199578,39cyclictest5122-21missed_timers08:06:050
1088199576,43cyclictest0-21swapper/008:12:200
10881995728,8cyclictest0-21swapper/005:15:550
1088199572,6cyclictest31293-21fschecks_count06:41:000
1088199572,38cyclictest11889-21grep06:56:030
10881995633,11cyclictest7850irq/56-dwc_otg_08:50:430
1088199562,8cyclictest15190-21aten_rbpower_po06:10:580
1088199561,41cyclictest20026-21apt-get04:55:490
1088199561,41cyclictest0-21swapper/008:31:120
10881995613,35cyclictest19293-21sh07:44:300
1088299553,4cyclictest0-21swapper/107:21:111
1088199552,5cyclictest0-21swapper/006:35:550
1088199552,5cyclictest0-21swapper/006:35:550
1088199552,38cyclictest0-21swapper/007:59:100
1088199552,37cyclictest18275-21/usr/sbin/munin09:41:080
1088199552,34cyclictest0-21swapper/006:46:010
1088199551,7cyclictest23005-21sed05:01:110
1088199551,4cyclictest11787-21wc04:31:090
1088199551,44cyclictest0-21swapper/009:36:040
10881995510,3cyclictest0-21swapper/005:06:010
203042540,3sleep320302-21grep04:56:003
10881995427,5cyclictest16128-21tr07:01:070
1088199541,42cyclictest0-21swapper/009:11:060
1088199541,41cyclictest0-21swapper/007:06:060
1088199541,41cyclictest0-21swapper/007:06:060
1088199531,5cyclictest0-21swapper/007:31:050
10881995312,34cyclictest22102-21diskmemload07:16:510
1088199529,34cyclictest0-21swapper/009:56:460
1088199529,31cyclictest0-21swapper/008:16:040
1088199521,38cyclictest18255-21aten_rbpower_vo04:50:580
1088199521,34cyclictest0-21swapper/005:10:560
10881995212,4cyclictest9450irq/48-DMA09:12:250
10881995212,30cyclictest923-21sshd06:01:400
91062516,10sleep00-21swapper/004:25:570
1088199518,38cyclictest0-21swapper/006:25:570
1088199512,36cyclictest0-21swapper/007:24:170
1088199511,9cyclictest13586-21fschecks_time06:06:010
1088199511,7cyclictest23561-21kworker/u8:205:40:480
1088199511,36cyclictest0-21swapper/009:16:070
1088199511,36cyclictest0-21swapper/009:16:070
1088199502,39cyclictest0-21swapper/005:26:040
10883994723,12cyclictest16015-21latency_hist07:40:432
1088199475,32cyclictest0-21swapper/008:55:000
1088199475,32cyclictest0-21swapper/008:55:000
10881994724,4cyclictest0-21swapper/005:35:570
1088199471,35cyclictest0-21swapper/004:46:090
10884994614,22cyclictest20807-21fschecks_time06:26:003
10883994620,7cyclictest27760-21/usr/sbin/munin07:16:052
1088199458,5cyclictest7750irq/56-dwc_otg05:55:420
10881994513,4cyclictest15592-21unixbench-2d04:41:090
236482440,2sleep30-21swapper/307:10:543
1088299447,24cyclictest0-21swapper/105:45:571
10883994321,11cyclictest16906-21aten_rbpower_en08:20:572
1088299433,30cyclictest0-21swapper/108:41:051
10882994321,4cyclictest17915-21idleruntime-cro04:50:431
239682420,2sleep20-21swapper/207:50:312
10884994214,17cyclictest25803-21ssh09:10:553
10882994210,12cyclictest0-21swapper/109:10:431
10884994123,5cyclictest25224-21idleruntime-cro05:10:433
10884994115,15cyclictest0-21swapper/308:31:023
10883994123,11cyclictest28757-21ssh06:37:292
10883994123,11cyclictest28757-21ssh06:37:292
326122400,1sleep10-21swapper/109:57:161
222142400,1sleep30-21swapper/307:08:453
222142400,1sleep30-21swapper/307:08:453
1088499407,7cyclictest7750irq/56-dwc_otg07:01:053
10884994018,14cyclictest14136-21/usr/sbin/munin09:35:543
10884994018,13cyclictest27898-21ssh07:15:573
1088399408,3cyclictest0-21swapper/207:14:292
10883994021,5cyclictest685-21sh06:42:392
1088299407,21cyclictest0-21swapper/107:55:591
1088299402,5cyclictest20279-21ntp_states07:06:061
1088299402,5cyclictest20279-21ntp_states07:06:061
10882994018,9cyclictest11853-21missed_timers06:56:031
10882994018,5cyclictest14314-21/usr/sbin/munin08:56:101
1088299401,22cyclictest0-21swapper/107:48:211
1088499399,5cyclictest0-21swapper/307:25:573
1088499397,16cyclictest0-21swapper/307:49:563
1088499391,5cyclictest30289-21ssh07:56:513
1088499391,23cyclictest7950irq/56-dwc_otg_06:31:003
1088499391,20cyclictest0-21swapper/307:30:543
10883993920,5cyclictest20469-21sh09:04:112
10883993919,13cyclictest8525-21ssh06:51:262
1088299393,23cyclictest0-21swapper/107:31:041
1088299392,29cyclictest7750irq/56-dwc_otg08:16:241
10882993918,5cyclictest19592-21sh08:23:481
1088299391,5cyclictest917-21snmpd08:40:381
1088299391,4cyclictest0-21swapper/107:51:051
133542380,2sleep21088399cyclictest09:34:552
1088499389,3cyclictest0-21swapper/309:06:023
1088499381,6cyclictest31046-21fschecks_time09:56:013
1088499381,22cyclictest22102-21diskmemload09:16:073
1088499381,22cyclictest22102-21diskmemload09:16:073
10883993817,12cyclictest908-21runrttasks08:09:532
10883993813,4cyclictest9674-21aten_rbpower_vo05:55:582
1088299385,26cyclictest7850irq/56-dwc_otg_04:50:571
1088299384,17cyclictest21-21rcuc/105:57:041
1088299383,19cyclictest0-21swapper/109:14:381
10882993819,17cyclictest473350ssh06:47:131
1088299381,27cyclictest0-21swapper/109:41:061
1088299381,25cyclictest10282-21fschecks_count09:31:001
313352370,2sleep1221ktimersoftd/106:41:011
1088499378,20cyclictest25292-21grep07:51:083
10884993719,5cyclictest14517-21/usr/sbin/munin04:41:043
1088499371,3cyclictest0-21swapper/305:01:113
1088499371,22cyclictest0-21swapper/305:45:573
10884993711,4cyclictest0-21swapper/308:01:053
10883993713,6cyclictest123492chrt04:34:222
1088399371,15cyclictest0-21swapper/209:45:232
1088299372,11cyclictest26757-21rm08:31:461
10882993720,7cyclictest30067-21sh09:55:111
10882993717,13cyclictest13071-21idleruntime-cro06:05:421
10882993714,5cyclictest23-21ksoftirqd/104:35:431
1088299371,3cyclictest0-21swapper/107:41:391
1088299371,23cyclictest23409-21ssh08:28:131
1088499366,23cyclictest8635-21ssh06:51:383
10884993619,9cyclictest16957-21open_files07:41:073
10884993618,11cyclictest381ktimersoftd/308:07:503
10884993616,5cyclictest13340-21sh09:34:533
10884993615,4cyclictest29845-21sh06:39:203
10884993615,4cyclictest29845-21sh06:39:203
10884993615,12cyclictest381ktimersoftd/309:02:273
1088499361,25cyclictest27261-21ls09:51:063
1088399369,4cyclictest32201-21sh08:00:202
10883993626,6cyclictest4052-21fschecks_time07:26:002
10883993618,5cyclictest22514-21aten_rbpower_en06:30:572
10883993618,4cyclictest5076-21/usr/sbin/munin08:46:082
10883993617,4cyclictest1461350wget04:40:552
10883993615,5cyclictest23413-21sh09:07:002
10883993614,5cyclictest26026-21sh09:10:592
10883993614,4cyclictest28797-21sh09:52:522
1088299368,13cyclictest9486-21apt-config05:55:431
1088299364,20cyclictest0-21swapper/108:12:101
1088299363,25cyclictest0-21swapper/106:20:401
1088299363,13cyclictest0-21swapper/108:50:401
1088299362,6cyclictest0-21swapper/105:21:101
10882993623,5cyclictest25888-21sh09:50:061
10882993622,4cyclictest12455-21sh08:54:401
10882993622,4cyclictest12455-21sh08:54:401
10882993621,4cyclictest3612-21sh07:25:441
10882993619,7cyclictest221ktimersoftd/109:35:561
1088299361,5cyclictest27495-21/usr/sbin/munin06:36:071
1088299361,5cyclictest27495-21/usr/sbin/munin06:36:071
1088299361,5cyclictest22096-21modprobe06:30:431
81192350,2sleep10-21swapper/106:51:111
1088499359,3cyclictest0-21swapper/307:24:413
1088499357,4cyclictest7850irq/56-dwc_otg_06:15:423
1088499355,4cyclictest0-21swapper/309:24:463
1088499354,25cyclictest0-21swapper/304:50:163
1088499354,19cyclictest0-21swapper/308:25:553
1088499354,17cyclictest0-21swapper/305:55:433
10884993516,5cyclictest6236-21if_wlan009:26:023
10884993514,13cyclictest381ktimersoftd/308:55:353
10884993514,13cyclictest381ktimersoftd/308:55:353
1088499351,3cyclictest0-21swapper/308:36:123
1088499351,24cyclictest0-21swapper/305:30:553
1088499351,23cyclictest0-21swapper/306:45:573
1088499351,22cyclictest0-21swapper/308:42:373
10884993510,12cyclictest12642-21unixbench_singl06:01:113
1088399355,9cyclictest7850irq/56-dwc_otg_09:50:432
1088399352,14cyclictest5765-21ssh09:25:552
1088399351,9cyclictest7850irq/56-dwc_otg_06:20:432
10883993519,4cyclictest26682-21sh08:31:392
10883993515,4cyclictest22797-21sh07:09:492
10883993515,4cyclictest22797-21sh07:09:492
1088299356,4cyclictest7850irq/56-dwc_otg_06:30:571
1088299352,3cyclictest0-21swapper/105:30:581
1088299351,23cyclictest0-21swapper/104:39:101
1088299351,19cyclictest20284-21munin-run06:25:431
1088499349,7cyclictest15910-21rm07:40:433
10884993418,5cyclictest16530-21idleruntime-cro08:20:433
1088499341,5cyclictest29992-21grep05:21:093
10884993415,5cyclictest8877-21sh08:50:433
10884993414,13cyclictest16844-21ssh09:00:083
10884993412,4cyclictest11201-21sh08:13:333
1088499341,22cyclictest28119-21ls05:16:093
1088499341,22cyclictest19786-21ls04:55:443
10884993410,16cyclictest19827-21ssh08:24:143
1088399349,9cyclictest32052-1kworker/u9:009:21:432
1088399348,4cyclictest0-21swapper/208:57:202
1088399346,4cyclictest7750irq/56-dwc_otg06:48:402
1088399344,13cyclictest7950irq/56-dwc_otg_04:51:012
10883993417,4cyclictest39512ssh08:44:152
10883993415,5cyclictest13591-21fschecks_time06:06:012
10883993414,5cyclictest15348-21/usr/sbin/munin07:01:032
10883993414,4cyclictest301ktimersoftd/208:01:042
1088299344,14cyclictest17489-21/usr/sbin/munin09:00:581
1088299343,4cyclictest0-21swapper/107:01:011
1088299341,5cyclictest3213850ssh09:17:501
1088299341,5cyclictest3213850ssh09:17:501
1088299341,5cyclictest0-21swapper/109:27:221
10882993412,4cyclictest0-21swapper/107:18:321
1088299341,15cyclictest0-21swapper/107:36:111
1088499338,4cyclictest0-21swapper/306:40:563
1088499331,21cyclictest17888-21timerwakeupswit06:16:103
10884993311,6cyclictest12711-21/usr/sbin/munin04:35:553
1088399336,15cyclictest0-21swapper/205:53:182
10883993310,5cyclictest7950irq/56-dwc_otg_05:35:572
1088299338,11cyclictest0-21swapper/108:06:481
1088299335,9cyclictest7750irq/56-dwc_otg05:26:041
1088299332,18cyclictest0-21swapper/108:04:201
1088299332,16cyclictest15275-21grep04:41:061
10882993315,9cyclictest3678-21rm09:22:331
1088299331,4cyclictest0-21swapper/105:16:051
1088299331,20cyclictest0-21swapper/104:55:491
1088299331,16cyclictest0-21swapper/106:06:021
1088499328,7cyclictest0-21swapper/306:56:193
1088499326,4cyclictest7750irq/56-dwc_otg09:41:063
1088499326,16cyclictest22787-21cut09:46:003
1088399325,19cyclictest7950irq/56-dwc_otg_08:40:272
1088399324,5cyclictest21040-21cut08:25:572
1088399324,13cyclictest21168-21unixbench_multi04:56:112
1088399323,6cyclictest0-21swapper/205:44:542
1088399321,7cyclictest7950irq/56-dwc_otg_09:36:112
10883993214,6cyclictest31098-21grep09:16:102
10883993214,6cyclictest31098-21grep09:16:102
10883993211,6cyclictest29985-21/usr/sbin/munin05:21:092
1088299324,14cyclictest9450irq/48-DMA09:12:251
1088299321,23cyclictest0-21swapper/105:38:411
1088299321,15cyclictest0-21swapper/105:40:481
319692310,3sleep231971-21grep05:26:102
1088499319,8cyclictest9597-21basename05:55:583
1088499315,20cyclictest0-21swapper/305:10:563
1088499312,5cyclictest10911-21apt-get04:30:543
1088499311,4cyclictest31854-21ls05:26:093
1088499311,19cyclictest20387-21ls06:25:433
1088399318,4cyclictest0-21swapper/207:51:092
1088399315,7cyclictest7750irq/56-dwc_otg07:44:012
1088399315,5cyclictest0-21swapper/208:53:172
1088399315,5cyclictest0-21swapper/208:53:172
10883993125,4cyclictest23720-21aten_rbpower_en05:05:572
10883993113,5cyclictest7950irq/56-dwc_otg_08:15:562
1088299311,19cyclictest0-21swapper/106:11:581
1088299311,17cyclictest23021-21sed05:01:111
1088499301,5cyclictest0-21swapper/306:06:573
1088499301,18cyclictest0-21swapper/305:36:133
1088399304,18cyclictest0-21swapper/205:14:352
1088399302,11cyclictest7950irq/56-dwc_otg_05:20:432
1088399301,6cyclictest11753-21ssh08:14:352
10883993010,9cyclictest21843-21/usr/sbin/munin05:00:592
1088299301,17cyclictest0-21swapper/105:13:041
1088399298,5cyclictest7950irq/56-dwc_otg_05:31:012
1088399297,12cyclictest0-21swapper/204:46:322
1088399293,5cyclictest15390-21cut06:11:012
1088399292,6cyclictest11500-21/usr/sbin/munin06:00:562
10883992911,7cyclictest142332chrt04:40:132
101212294,12sleep20-21swapper/204:26:112
1088499282,4cyclictest3625-21apt-get05:40:483
1088399288,4cyclictest7950irq/56-dwc_otg_06:20:552
1088399282,4cyclictest7850irq/56-dwc_otg_06:25:572
1088399281,4cyclictest7950irq/56-dwc_otg_05:46:112
90482273,16sleep10-21swapper/104:25:461
103262273,15sleep30-21swapper/304:27:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional