You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 08:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 03, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2119921620,10sleep31546299cyclictest23:40:313
154599913413,99cyclictest13897-21netstat00:10:390
15459991334,101cyclictest12979-21sed00:10:160
154599913133,66cyclictest8006-21unixbench_multi21:25:440
15459991282,104cyclictest27985-21latency21:50:400
1545999128115,8cyclictest7850irq/56-dwc_otg_21:35:300
15459991265,18cyclictest0-21swapper/022:20:390
154599912634,67cyclictest0-21swapper/022:35:400
154599912619,80cyclictest24300-21timerandwakeup21:45:420
2347021230,6sleep31546299cyclictest22:25:153
154609911954,36cyclictest20669-21cat22:20:411
15459991182,31cyclictest28828-21ssh22:30:400
15459991133,17cyclictest21204-21diskstats23:40:320
154609910829,29cyclictest28000-21taskset21:50:401
154599910819,64cyclictest0-21swapper/022:50:160
154609910633,46cyclictest7750irq/56-dwc_otg00:10:151
154629910530,45cyclictest13839-21sed00:10:383
154609910431,52cyclictest11702-21http22:50:161
154609910317,56cyclictest0-21swapper/100:25:211
1545999993,55cyclictest28660-21ssh23:10:330
1545999974,67cyclictest27805-21switchtime21:10:420
15460999620,36cyclictest0-21swapper/100:10:371
15462999531,44cyclictest24375-21ssh21:45:433
15462999510,55cyclictest0-21swapper/300:25:213
15462999433,17cyclictest28334-21/usr/sbin/munin21:50:473
15459999264,7cyclictest7850irq/56-dwc_otg_21:30:370
1545999926,20cyclictest26459-21diskmemload00:30:340
100072900,2sleep10-21swapper/122:47:111
15460998926,18cyclictest11502-21memory21:30:361
15459998855,17cyclictest7750irq/56-dwc_otg00:25:220
15460998629,37cyclictest7750irq/56-dwc_otg21:45:421
1546299853,48cyclictest12993-21ls00:10:163
15460998427,32cyclictest30094-21cut00:30:341
15460998328,28cyclictest7950irq/56-dwc_otg_21:10:411
196012820,2sleep09-21ksoftirqd/022:59:300
1545999821,29cyclictest9412-21wc23:25:400
15461998017,34cyclictest0-21swapper/221:25:442
1546099795,44cyclictest7850irq/56-dwc_otg_21:25:441
15460997927,18cyclictest416-21ssh22:35:371
129122790,13sleep31546299cyclictest22:50:393
1545999782,5cyclictest1323-21wget20:00:280
1545999773,17cyclictest0-21swapper/019:20:260
1545999771,30cyclictest22946-21latency_hist21:45:150
1546099766,50cyclictest21422-21if_eth023:40:371
1545999761,70cyclictest20430-21aten_rbpower_po23:00:280
1545999761,23cyclictest18281-21grep00:15:420
159972750,2sleep20-21swapper/222:15:312
15462997530,17cyclictest8067-21users21:25:453
1546299743,49cyclictest0-21swapper/300:30:333
15460997429,27cyclictest28782-21ntp_states22:30:391
15460997410,43cyclictest0-21swapper/121:35:301
15459997420,5cyclictest0-21swapper/019:25:250
15461997216,19cyclictest301ktimersoftd/222:20:402
15459997218,4cyclictest15246-21modprobe22:15:140
15459997212,11cyclictest0-21swapper/023:08:500
1545999711,5cyclictest32714-21ssh21:55:430
1545999703,35cyclictest0-21swapper/019:55:280
1545999703,35cyclictest0-21swapper/019:55:280
1545999701,21cyclictest0-21swapper/000:01:560
15462996940,23cyclictest32087-21/usr/sbin/munin22:35:393
1545999692,25cyclictest0-21swapper/022:49:100
1545999691,4cyclictest31067-21aten_rbpower_po21:15:310
15462996840,22cyclictest11505-21memory21:30:373
1545999682,24cyclictest0-21swapper/000:21:210
1545999681,25cyclictest0-21swapper/023:22:530
15461996716,21cyclictest7850irq/56-dwc_otg_21:30:372
1545999671,5cyclictest4975-21grep22:40:440
15459996616,4cyclictest0-21swapper/019:45:280
147802660,2sleep20-21swapper/223:32:092
288572650,2sleep00-21swapper/023:50:150
15462996533,24cyclictest27720-21zcat21:10:403
1545999652,23cyclictest0-21swapper/023:17:150
15459996515,11cyclictest0-21swapper/020:43:270
1545999651,15cyclictest0-21swapper/022:05:380
1545999651,15cyclictest0-21swapper/022:05:380
15462996436,23cyclictest28779-21sh22:30:373
1546199642,14cyclictest7750irq/56-dwc_otg23:50:282
15461996416,20cyclictest0-21swapper/200:25:422
1545999644,18cyclictest0-21swapper/022:17:000
15462996332,26cyclictest15003-21ssh21:35:283
1546199638,33cyclictest27836-21ls21:10:422
1546199636,14cyclictest0-21swapper/200:05:372
15461996326,11cyclictest28374-21ssh21:50:492
1545999634,12cyclictest0-21swapper/000:38:580
1545999632,27cyclictest0-21swapper/020:48:450
1545999632,19cyclictest0-21swapper/023:53:350
15459996314,32cyclictest0-21swapper/019:15:420
15459996311,15cyclictest0-21swapper/021:20:560
15459996311,13cyclictest0-21swapper/019:41:390
1546199629,12cyclictest24375-21ssh21:45:432
1545999627,15cyclictest0-21swapper/023:33:260
1545999622,6cyclictest15455-1kworker/0:1H20:30:150
15459996213,32cyclictest6811-21aten_rbpower_po20:15:280
1546199615,9cyclictest0-21swapper/219:15:252
1546199614,37cyclictest30130-21fschecks_time00:30:342
15461996114,26cyclictest7950irq/56-dwc_otg_22:50:392
1545999611,20cyclictest20265-21sh23:39:380
1545999611,20cyclictest20265-21sh23:39:380
15460996024,14cyclictest1441-21ssh00:35:271
1545999601,21cyclictest0-21swapper/020:54:270
15460995919,21cyclictest31861-21sh23:14:171
1545999594,16cyclictest15452-21cyclictest22:27:550
1545999594,16cyclictest15452-21cyclictest22:27:550
1545999592,25cyclictest0-21swapper/020:35:200
15459995910,30cyclictest6026-21grep20:10:410
15460995816,16cyclictest1904-21/usr/sbin/munin20:00:381
15460995812,26cyclictest15816-21ssh22:15:301
1545999583,31cyclictest0-21swapper/000:00:150
15459995811,15cyclictest0-21swapper/020:24:360
1546299578,11cyclictest7950irq/56-dwc_otg_21:00:273
1546299578,11cyclictest7950irq/56-dwc_otg_21:00:273
15462995718,18cyclictest23680-21cat19:30:393
1545999576,20cyclictest0-21swapper/021:05:260
15462995623,13cyclictest18201-21/usr/sbin/munin00:15:413
15461995632,15cyclictest0-21swapper/221:35:312
1546099567,37cyclictest7850irq/56-dwc_otg_22:05:261
1546099567,37cyclictest7850irq/56-dwc_otg_22:05:261
1546099567,30cyclictest0-21swapper/121:20:301
15460995612,21cyclictest7850irq/56-dwc_otg_21:15:291
1545999569,13cyclictest0-21swapper/019:50:260
151292563,16sleep10-21swapper/119:09:291
15462995517,25cyclictest24473-21taskset00:23:563
1546199553,11cyclictest21317-21perl23:40:352
1546099559,30cyclictest0-21swapper/120:10:211
1545999552,30cyclictest22420-21sed20:55:410
43962540,2sleep30-21swapper/323:59:523
1546199543,11cyclictest0-21swapper/220:05:322
15460995414,20cyclictest18088-21/usr/sbin/munin00:15:391
15460995411,22cyclictest12028-21idleruntime-cro20:30:141
1545999541,37cyclictest1180-21runrttasks19:35:400
15462995323,15cyclictest8374-21ssh00:04:363
15462995321,20cyclictest31040-21aten_rbpower_en21:15:303
15462995320,21cyclictest29958-21ntp_states23:50:393
15462995320,19cyclictest689-21taskset21:56:223
15462995312,25cyclictest7950irq/56-dwc_otg_19:25:413
1546099539,26cyclictest7850irq/56-dwc_otg_22:25:421
1546099539,26cyclictest7850irq/56-dwc_otg_22:25:421
15460995319,7cyclictest14610-21ls20:35:351
1545999531,12cyclictest23165-21ls19:30:320
275272520,2sleep310117-21kworker/3:123:08:553
15462995224,20cyclictest6302-21ssh22:04:023
15462995220,11cyclictest0-21swapper/320:00:253
15462995217,12cyclictest4102-21awk23:20:143
15462995214,22cyclictest7950irq/56-dwc_otg_20:30:153
1546199522,9cyclictest7850irq/56-dwc_otg_22:30:262
15461995215,12cyclictest431-21ssh22:35:392
15461995212,23cyclictest31-21ksoftirqd/200:10:362
15460995211,22cyclictest30218-21sed23:50:421
1545999521,31cyclictest23209-21wget21:00:290
1545999521,31cyclictest23209-21wget21:00:290
1546299513,6cyclictest7950irq/56-dwc_otg_20:30:273
15462995118,17cyclictest5448-21sh21:23:283
1546299511,6cyclictest7950irq/56-dwc_otg_20:15:263
15462995111,19cyclictest7950irq/56-dwc_otg_23:34:003
1546099517,20cyclictest7850irq/56-dwc_otg_23:35:411
1546099517,20cyclictest7850irq/56-dwc_otg_23:35:411
1546099512,7cyclictest7750irq/56-dwc_otg20:30:271
1546299508,26cyclictest0-21swapper/320:10:213
15462995021,13cyclictest20827-21latency_hist19:25:153
15462995019,12cyclictest10707-21ssh22:09:333
15462995019,12cyclictest10707-21ssh22:09:333
15462995017,11cyclictest9650-21ssh22:46:323
1546199503,12cyclictest17293-21ssh00:15:272
1546099502,28cyclictest0-21swapper/119:55:281
1546099502,28cyclictest0-21swapper/119:55:281
15460995017,14cyclictest6543-21ssh22:04:291
1545999502,25cyclictest0-21swapper/020:30:300
1546299494,13cyclictest7950irq/56-dwc_otg_19:45:283
15462994922,9cyclictest3934-21ls20:05:453
15462994920,7cyclictest8785-21df23:25:303
15462994918,12cyclictest3860-21ssh00:37:453
1546199492,10cyclictest25152-21uniq21:05:322
15460994918,12cyclictest22972-21ssh23:03:071
15460994914,15cyclictest21317-21ssh21:42:191
15460994912,22cyclictest16336-21kernelversion20:40:351
15460994911,21cyclictest7750irq/56-dwc_otg22:43:311
15462994818,21cyclictest7309-21ssh23:23:363
15462994815,17cyclictest25144-21sh23:45:283
15462994814,11cyclictest21231-21/usr/sbin/munin23:00:383
15462994813,22cyclictest26852-21ssh22:28:563
15462994813,22cyclictest26852-21ssh22:28:563
15462994813,18cyclictest7950irq/56-dwc_otg_20:40:413
15462994811,25cyclictest13828-21taskset22:12:423
1546099489,25cyclictest0-21swapper/119:40:281
15460994819,12cyclictest13156-21ssh22:11:291
15460994816,7cyclictest21708-21ntp_states19:25:381
15460994815,19cyclictest4158-21sh23:59:251
15460994814,18cyclictest8801-21sh23:25:311
15460994814,18cyclictest18502-21ssh22:57:281
1546099478,26cyclictest0-21swapper/119:35:281
15460994713,14cyclictest25172-21cut23:45:291
15460994710,17cyclictest0-21swapper/100:00:411
15459994723,12cyclictest0-21swapper/022:00:280
15462994618,12cyclictest31893-21ssh23:14:203
1546199462,12cyclictest0-21swapper/219:25:252
1546099462,22cyclictest0-21swapper/119:45:281
15460994616,9cyclictest468-21ssh21:55:591
15460994615,21cyclictest22805-21/usr/sbin/munin19:30:441
15462994519,15cyclictest5974-21ssh22:42:193
15462994518,12cyclictest16712-21cat22:55:343
15462994515,11cyclictest14622-21missed_timers20:35:363
15462994514,19cyclictest21346-21taskset21:42:233
15462994513,18cyclictest25857-21chrt19:36:073
15462994513,12cyclictest20229-21ssh23:39:343
15462994513,12cyclictest20229-21ssh23:39:343
15462994511,12cyclictest15537-21awk22:15:233
1546099458,20cyclictest0-21swapper/120:46:231
1546099458,12cyclictest4072-21ls20:05:461
15460994519,16cyclictest14413-21ssh23:31:281
15460994515,12cyclictest27648-21ssh23:09:081
15460994514,16cyclictest20723-21sshd20:51:191
150512453,13sleep00-21swapper/019:08:270
15460994413,18cyclictest31579-21seq19:55:041
15460994413,12cyclictest26459-21diskmemload23:22:421
15460994412,20cyclictest3936-21sh23:20:001
15459994415,15cyclictest2993-21dhclient-script20:05:190
1545999441,17cyclictest0-21swapper/019:15:150
1546199437,24cyclictest0-21swapper/219:20:282
15460994317,9cyclictest0-21swapper/119:20:261
15460994313,14cyclictest22760-21runrttasks20:59:201
15460994310,21cyclictest24712-21sh00:24:231
1546199427,25cyclictest0-21swapper/220:35:292
1546099426,11cyclictest16794-21chrt19:12:101
15462994112,15cyclictest7950irq/56-dwc_otg_19:57:483
15462994112,15cyclictest7950irq/56-dwc_otg_19:57:483
15462994110,15cyclictest7950irq/56-dwc_otg_19:55:073
1546199417,24cyclictest0-21swapper/220:10:262
1546199417,22cyclictest0-21swapper/221:40:282
1546099417,23cyclictest0-21swapper/120:20:281
1546299409,11cyclictest7850irq/56-dwc_otg_20:48:543
15462994011,6cyclictest18373-21ls19:15:413
15462993911,8cyclictest27260-21smartpower19:40:403
1546199397,24cyclictest0-21swapper/223:15:262
15461993916,4cyclictest7750irq/56-dwc_otg22:40:312
15460993911,15cyclictest0-21swapper/121:05:281
15462993825,11cyclictest15920-21if_wlan019:10:343
15462993722,11cyclictest9674-21grep20:20:423
15462993711,10cyclictest0-21swapper/320:54:013
15462993622,11cyclictest25286-21if_err_wlan021:05:333
15462993612,11cyclictest22783-21chrt20:59:393
15461993625,4cyclictest7950irq/56-dwc_otg_00:05:152
1546199361,24cyclictest18203-21ls20:45:342
15460993626,8cyclictest24142-21ls21:00:431
15460993626,8cyclictest24142-21ls21:00:431
214232351,4sleep221497-21df_inode20:55:292
1546199357,4cyclictest0-21swapper/222:13:032
1546199353,23cyclictest7850irq/56-dwc_otg_21:20:332
15461993526,6cyclictest3469-21uniq22:00:322
15461993519,6cyclictest31792-21ssh21:15:422
1546199351,6cyclictest7750irq/56-dwc_otg20:20:322
1546199351,19cyclictest0-21swapper/220:30:412
1546199347,10cyclictest7850irq/56-dwc_otg_22:25:252
1546199347,10cyclictest7850irq/56-dwc_otg_22:25:252
1546199346,16cyclictest7950irq/56-dwc_otg_22:05:342
1546199346,16cyclictest7950irq/56-dwc_otg_22:05:342
1546199345,20cyclictest7850irq/56-dwc_otg_00:00:162
1546199342,19cyclictest7750irq/56-dwc_otg23:45:292
1546199341,8cyclictest29903-21ssh23:10:482
1546199341,21cyclictest20805-21ls23:40:162
1546199341,21cyclictest20805-21ls23:40:162
1546099347,15cyclictest7950irq/56-dwc_otg_19:16:431
1546099345,13cyclictest0-21swapper/120:16:221
1546199339,11cyclictest7950irq/56-dwc_otg_23:05:432
1546199337,6cyclictest7850irq/56-dwc_otg_22:55:432
1546199335,5cyclictest7850irq/56-dwc_otg_19:10:292
1546199332,15cyclictest0-21swapper/223:25:372
15461993314,7cyclictest31796-21/usr/sbin/munin21:55:322
15461993314,5cyclictest32658-21/usr/sbin/munin19:55:402
15461993314,5cyclictest32658-21/usr/sbin/munin19:55:402
15461993312,5cyclictest27489-21grep19:40:422
1546199327,5cyclictest7750irq/56-dwc_otg00:20:272
15461993215,4cyclictest1180-21runrttasks20:15:282
15461993210,12cyclictest7950irq/56-dwc_otg_20:00:432
1546199316,6cyclictest7750irq/56-dwc_otg23:20:302
1546199316,12cyclictest7950irq/56-dwc_otg_21:00:442
1546199316,12cyclictest7950irq/56-dwc_otg_21:00:442
1546199315,11cyclictest0-21swapper/219:30:272
1546199313,4cyclictest1695-21/usr/sbin/munin00:35:322
15461993113,4cyclictest16984-21grep20:40:432
1546199305,5cyclictest7850irq/56-dwc_otg_22:45:432
1546199301,6cyclictest2132950ssh23:00:402
1546199294,12cyclictest7950irq/56-dwc_otg_19:35:202
15461992922,4cyclictest30103-21aten_rbpower_en19:50:292
148862294,17sleep30-21swapper/319:06:233
136322294,14sleep20-21swapper/219:05:252
15461992822,4cyclictest10509-21aten_rbpower_en20:25:272
1546199281,16cyclictest19834-21ls20:50:322
1546199271,15cyclictest28475-21cut19:45:342
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional