You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 13:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Dec 09, 2025 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2004722570,2sleep30-21swapper/319:05:293
2201021730,2sleep02182199cyclictest21:37:200
21821998072,2cyclictest0-21swapper/023:06:060
170712770,1sleep20-21swapper/220:25:362
245512750,4sleep324554-21missed_timers20:45:383
78342740,5sleep37837-21fschecks_count20:00:343
21821997412,46cyclictest101ktimersoftd/022:20:150
2182199736,40cyclictest0-21swapper/020:25:300
21821997035,12cyclictest0-21swapper/000:35:160
21821996810,52cyclictest101ktimersoftd/023:25:150
2182199673,31cyclictest0-21swapper/019:10:180
2182199673,31cyclictest0-21swapper/019:10:180
2182399666,19cyclictest7850irq/56-dwc_otg_23:15:312
2182199664,5cyclictest101ktimersoftd/023:30:430
2182199662,40cyclictest0-21swapper/022:30:290
21821996555,7cyclictest0-21swapper/023:25:300
204362650,2sleep30-21swapper/321:35:373
21821996438,10cyclictest101ktimersoftd/022:00:320
21821996434,19cyclictest101ktimersoftd/019:55:150
21821996345,4cyclictest7850irq/56-dwc_otg_23:00:260
2182199613,30cyclictest0-21swapper/021:15:280
304732600,2sleep20-21swapper/221:48:492
21821996024,10cyclictest0-21swapper/020:55:330
2182199602,39cyclictest18396-21/usr/sbin/munin20:30:330
21821996014,43cyclictest12942-21latency_hist22:50:160
2182199601,40cyclictest0-21swapper/020:45:210
21821996012,33cyclictest30039-21unixbench_multi00:35:440
213832602,51sleep10-21swapper/119:08:071
2182199596,39cyclictest0-21swapper/019:55:210
2182199596,39cyclictest0-21swapper/019:55:210
21821995945,3cyclictest0-21swapper/021:46:190
21821995937,5cyclictest15038-21ssh00:15:430
21821995927,17cyclictest0-21swapper/022:10:280
2182199592,36cyclictest11025-21fschecks_count22:05:320
2182199587,25cyclictest0-21swapper/000:25:280
21821995820,8cyclictest32412-21sh21:50:410
2182199581,35cyclictest24681-21/usr/sbin/munin21:40:410
2182199581,35cyclictest24681-21/usr/sbin/munin21:40:410
2182199581,17cyclictest0-21swapper/023:15:310
2182199581,11cyclictest0-21swapper/019:20:280
21821995810,26cyclictest6336-21grep22:40:420
2182199578,25cyclictest0-21swapper/020:20:270
21821995733,5cyclictest9458-21ls21:20:450
21821995717,18cyclictest3793-21latency_hist19:50:160
2182199563,39cyclictest22339-21/usr/sbin/munin22:20:260
21821995629,5cyclictest7850irq/56-dwc_otg_21:55:210
325542550,2sleep20-21swapper/223:56:152
21821995535,10cyclictest0-21swapper/023:39:160
2182199552,31cyclictest0-21swapper/021:10:320
2182199552,31cyclictest0-21swapper/021:10:320
2182199545,34cyclictest21431-21chrt20:36:050
198072540,2sleep30-21swapper/322:15:473
3672530,2sleep20-21swapper/221:50:462
21821995338,4cyclictest14268-21taskset20:16:530
21821995331,4cyclictest7950irq/56-dwc_otg_22:26:420
21821995329,4cyclictest2408-21/usr/sbin/munin00:00:310
2182199531,43cyclictest12290-21unixbench_multi20:10:430
218152530,5sleep20-21swapper/219:10:172
218152530,5sleep20-21swapper/219:10:172
249002520,2sleep3381ktimersoftd/323:46:223
21821995229,10cyclictest0-21swapper/023:50:270
2182199522,28cyclictest30972-21/usr/sbin/munin23:55:420
21821995221,15cyclictest29620-21fschecks_time21:00:350
2182199521,30cyclictest13285-21ls21:25:430
2182199521,28cyclictest0-21swapper/023:10:430
2182199521,28cyclictest0-21swapper/023:10:430
2182399517,14cyclictest0-21swapper/219:20:282
2182199518,10cyclictest7850irq/56-dwc_otg_23:45:280
21821995133,10cyclictest0-21swapper/000:24:060
21821995133,10cyclictest0-21swapper/000:24:060
21821995129,5cyclictest101ktimersoftd/023:00:150
21821995126,5cyclictest6725-21grep00:05:330
2182199505,34cyclictest0-21swapper/020:00:410
21821995033,4cyclictest21351-21ssh23:42:030
21821995024,3cyclictest0-21swapper/019:30:270
2182199501,24cyclictest26894-21unixbench_singl20:50:430
21822994912,19cyclictest0-21swapper/119:10:181
21822994912,19cyclictest0-21swapper/119:10:181
2182199499,37cyclictest101ktimersoftd/020:05:450
2182199499,18cyclictest0-21swapper/022:35:410
2182199499,18cyclictest0-21swapper/022:35:410
21821994931,11cyclictest0-21swapper/019:25:250
21821994926,11cyclictest0-21swapper/021:35:000
21821994922,15cyclictest1116-21cat19:40:420
21821994916,16cyclictest10474-21ssh00:10:330
2182299487,3cyclictest0-21swapper/119:20:281
21822994821,5cyclictest653550wget00:05:311
2182499473,26cyclictest0-21swapper/319:10:183
2182499473,26cyclictest0-21swapper/319:10:183
2182299477,5cyclictest0-21swapper/120:25:301
2182299475,5cyclictest32466-21ssh23:15:301
2182199475,20cyclictest0-21swapper/022:55:150
21821994717,22cyclictest0-21swapper/019:16:580
2182399467,25cyclictest0-21swapper/219:25:282
2182399467,25cyclictest0-21swapper/200:00:292
21822994616,5cyclictest21-21rcuc/123:30:361
2182199465,19cyclictest0-21swapper/020:40:290
21821994623,5cyclictest9-21ksoftirqd/021:05:340
2182199451,41cyclictest101ktimersoftd/019:38:310
2182399447,25cyclictest0-21swapper/223:35:292
2182399446,25cyclictest0-21swapper/222:45:292
21823994420,5cyclictest24888-21latency_hist00:30:152
21822994423,4cyclictest3438-21sh00:00:431
21823994316,9cyclictest0-21swapper/223:25:152
2182299435,8cyclictest23-21ksoftirqd/122:11:241
212862432,12sleep00-21swapper/019:06:520
2182499429,6cyclictest37-21rcuc/322:14:293
2182399426,22cyclictest0-21swapper/219:15:312
2182399425,27cyclictest7950irq/56-dwc_otg_19:45:292
21822994218,12cyclictest14602-21sh22:51:141
2182399417,21cyclictest0-21swapper/221:20:312
21822994121,6cyclictest22331-21sh22:20:261
21822994110,13cyclictest7950irq/56-dwc_otg_23:10:441
21822994110,13cyclictest7950irq/56-dwc_otg_23:10:441
21824994018,11cyclictest381ktimersoftd/323:25:323
2182399407,23cyclictest0-21swapper/221:35:312
2182299409,7cyclictest446-21cron23:25:151
2182299407,4cyclictest7850irq/56-dwc_otg_22:15:421
2182299403,6cyclictest8644-21grep20:00:441
2182299402,7cyclictest18119-21grep22:55:461
21822994012,7cyclictest9961-21tr22:45:381
21822994011,5cyclictest25169-21ssh00:30:281
210042400,3sleep10-21swapper/100:24:491
210042400,3sleep10-21swapper/100:24:491
21824993917,4cyclictest23585-21latency_hist20:45:163
2182399397,26cyclictest0-21swapper/220:15:292
2182399397,21cyclictest22367-21ssh22:20:292
2182399396,24cyclictest0-21swapper/220:05:312
21823993916,5cyclictest29762-21latency_hist22:30:152
2182299398,4cyclictest0-21swapper/121:32:111
2182299392,3cyclictest0-21swapper/121:25:281
21824993818,11cyclictest4528-21sh00:02:243
2182399387,21cyclictest0-21swapper/219:55:222
2182399387,21cyclictest0-21swapper/219:55:222
2182399382,25cyclictest13742-21ls22:50:392
21823993820,7cyclictest0-21swapper/221:15:382
21823993811,10cyclictest31-21ksoftirqd/223:30:412
2182299389,7cyclictest969-21fschecks_time21:10:361
2182299389,7cyclictest969-21fschecks_time21:10:361
2182299389,4cyclictest0-21swapper/122:28:541
2182299389,17cyclictest7950irq/56-dwc_otg_21:55:141
21822993831,5cyclictest7950irq/56-dwc_otg_00:10:291
2182299382,24cyclictest21948-21ssh00:25:361
21822993818,5cyclictest23399-21ssh21:40:081
21822993814,4cyclictest23656-21apt-config20:45:161
21822993813,6cyclictest0-21swapper/123:44:581
21822993812,5cyclictest28021-21fschecks_time21:45:341
21824993719,9cyclictest29693-21sh21:47:153
21824993712,15cyclictest32708-21diskmemload22:31:293
2182299375,7cyclictest13300-21cut20:15:331
21822993721,5cyclictest296192sleep100:35:411
112602370,2sleep10-21swapper/122:05:351
21824993622,4cyclictest27633-21ssh22:25:593
21824993617,8cyclictest39-21ksoftirqd/322:00:373
21824993613,15cyclictest0-21swapper/323:05:413
2182499361,28cyclictest20500-21cut20:35:333
2182499361,25cyclictest29028-21ls00:35:313
2182399361,5cyclictest7750irq/56-dwc_otg21:10:312
2182399361,5cyclictest7750irq/56-dwc_otg21:10:312
2182399361,13cyclictest19559-21sed20:30:432
2182299369,5cyclictest15552-21sh23:35:281
2182299369,20cyclictest0-21swapper/120:55:441
21822993619,4cyclictest566-21sh22:34:071
2182299361,7cyclictest540-21grep19:40:341
2182299361,29cyclictest0-21swapper/121:56:421
21822993612,7cyclictest6830-21sh22:41:041
2182299361,27cyclictest0-21swapper/123:04:101
2182299361,20cyclictest221ktimersoftd/120:05:451
2182499351,6cyclictest0-21swapper/322:43:203
21824993515,5cyclictest13791-21latency_hist00:15:163
21824993514,13cyclictest60492chrt23:22:343
2182399357,14cyclictest7950irq/56-dwc_otg_21:30:152
2182399352,24cyclictest0-21swapper/223:10:322
2182399352,24cyclictest0-21swapper/223:10:322
21823993514,4cyclictest19704-21latency_hist21:35:162
2182399351,23cyclictest11710-21ssh22:05:412
21823993510,3cyclictest0-21swapper/223:00:412
21822993516,5cyclictest21774-21latency_hist20:40:161
21822993514,5cyclictest27004-21cat23:50:151
21822993514,12cyclictest221ktimersoftd/123:50:371
2182499347,19cyclictest0-21swapper/300:09:273
2182499341,9cyclictest23556-21unixbench_multi22:20:443
21824993413,10cyclictest10063-21ps22:45:413
2182499341,26cyclictest12568-21switchtime23:30:423
2182499341,22cyclictest11369-21ls20:10:313
2182399347,24cyclictest0-21swapper/219:35:282
2182399344,20cyclictest29955-21ls00:35:432
21823993413,5cyclictest30141-21grep19:30:442
2182399341,25cyclictest20383-21ls20:35:312
2182399341,18cyclictest9317-21ls20:05:152
2182299349,18cyclictest0-21swapper/123:26:261
2182299342,7cyclictest4594-21aten_rbpower_vo21:15:311
2182299341,6cyclictest2083-21aten_rbpower_cu19:45:291
21822993414,9cyclictest27888-21sh23:09:481
21822993412,6cyclictest0-21swapper/120:50:401
21822993412,13cyclictest30358-21taskset19:31:531
21822993410,6cyclictest0-21swapper/119:35:431
149592340,4sleep121-21rcuc/100:15:431
2182499334,19cyclictest10766-21ssh22:05:273
2182499332,21cyclictest0-21swapper/321:10:353
2182499332,21cyclictest0-21swapper/321:10:353
21824993320,3cyclictest0-21swapper/322:35:383
21824993320,3cyclictest0-21swapper/322:35:383
21824993317,5cyclictest14198-21basename00:15:323
21824993314,4cyclictest32708-21diskmemload21:23:133
21824993314,4cyclictest15903-21latency_hist21:30:163
21823993318,4cyclictest1120-21runrttasks22:44:572
2182399331,24cyclictest18013-21ssh22:55:452
2182399331,23cyclictest7950irq/56-dwc_otg_23:43:512
2182399331,20cyclictest25762-21wget20:50:292
2182299333,7cyclictest2379950ssh21:40:281
2182299333,7cyclictest2379950ssh21:40:281
2182299331,8cyclictest31414-21ssh23:55:351
21822993315,4cyclictest19419-21timerwakeupswit20:30:421
21822993311,5cyclictest11700-21sort20:10:361
21822993311,4cyclictest24622-21sed19:15:441
2182499329,6cyclictest0-21swapper/321:55:403
2182499329,12cyclictest0-21swapper/322:56:473
2182499328,6cyclictest7850irq/56-dwc_otg_19:45:293
2182499325,4cyclictest7950irq/56-dwc_otg_20:00:143
2182499325,4cyclictest7950irq/56-dwc_otg_20:00:143
2182499321,6cyclictest5424-21munin-run19:55:153
21824993216,5cyclictest30895-21latency_hist21:05:153
2182399329,5cyclictest28294-21sort23:10:162
2182399329,5cyclictest1120-21runrttasks23:53:522
2182399328,6cyclictest7750irq/56-dwc_otg23:50:152
2182399328,4cyclictest7750irq/56-dwc_otg23:25:312
2182399326,13cyclictest7950irq/56-dwc_otg_19:55:152
2182399321,21cyclictest0-21swapper/200:14:472
2182299328,10cyclictest23-21ksoftirqd/121:20:351
2182299325,14cyclictest7950irq/56-dwc_otg_22:35:311
2182299325,14cyclictest7950irq/56-dwc_otg_22:35:311
21822993214,5cyclictest5193-21taskset19:52:321
21822993214,5cyclictest28768-21taskset19:29:311
21822993211,4cyclictest23-21ksoftirqd/120:25:151
2182499319,6cyclictest0-21swapper/323:42:353
2182499318,5cyclictest7950irq/56-dwc_otg_23:10:273
2182499318,5cyclictest7950irq/56-dwc_otg_23:10:273
2182499318,5cyclictest21393-21cat23:00:363
2182499318,15cyclictest0-21swapper/322:55:053
2182499317,15cyclictest0-21swapper/323:36:193
2182499313,13cyclictest7950irq/56-dwc_otg_00:25:353
21824993116,7cyclictest21277-1kworker/u9:123:58:423
21824993112,4cyclictest0-21swapper/300:20:343
21824993112,4cyclictest0-21swapper/300:20:343
2182499311,13cyclictest7750irq/56-dwc_otg21:15:343
2182399316,12cyclictest7950irq/56-dwc_otg_00:20:162
2182399313,17cyclictest576-21grep19:40:342
2182399313,17cyclictest3277-21diskstats21:55:312
2182399312,5cyclictest23662-21latency_hist20:45:162
2182399311,5cyclictest25148-21acpi00:30:272
2182399311,25cyclictest24094-21ls20:45:312
2182399311,19cyclictest24424-21ls21:40:372
2182399311,19cyclictest24424-21ls21:40:372
2182299314,13cyclictest7950irq/56-dwc_otg_21:05:311
2182299314,12cyclictest0-21swapper/122:01:071
2182299314,10cyclictest0-21swapper/119:55:291
2182299314,10cyclictest0-21swapper/119:55:291
21822993114,6cyclictest7850irq/56-dwc_otg_21:00:311
21824993024,4cyclictest27329-21aten_rbpower_en19:25:283
2182499301,5cyclictest24793-21head19:15:463
21824993011,12cyclictest27114-21taskset00:32:143
21824993011,11cyclictest381ktimersoftd/321:50:273
2182399305,17cyclictest7950irq/56-dwc_otg_22:35:312
2182399305,17cyclictest7950irq/56-dwc_otg_22:35:312
2182399302,10cyclictest0-21swapper/200:20:292
2182399302,10cyclictest0-21swapper/200:20:292
2182399301,6cyclictest17071-21ssh22:13:032
21823993010,11cyclictest19368-21sshd22:15:422
21822993023,5cyclictest0-21swapper/120:45:221
2182499296,15cyclictest381ktimersoftd/323:50:163
2182499296,13cyclictest0-21swapper/321:33:473
2182499291,21cyclictest0-21swapper/321:44:593
2182499291,21cyclictest0-21swapper/321:44:593
2182399297,4cyclictest0-21swapper/222:00:322
2182399292,6cyclictest0-21swapper/220:58:082
2182399292,17cyclictest0-21swapper/221:00:452
2182399292,17cyclictest0-21swapper/220:20:162
2182399291,6cyclictest30817-21ntp_states22:30:402
2182399291,13cyclictest31331-21snmpd00:07:012
2182499285,6cyclictest7850irq/56-dwc_otg_20:20:333
2182499282,16cyclictest7950irq/56-dwc_otg_20:29:113
2182499281,7cyclictest31635-21wc19:35:403
2182499281,6cyclictest514-21grep19:40:333
2182499281,5cyclictest7950irq/56-dwc_otg_20:55:313
2182499281,5cyclictest0-21swapper/321:05:303
21824992814,7cyclictest7850irq/56-dwc_otg_23:15:293
2182499281,15cyclictest7950irq/56-dwc_otg_19:35:153
202352283,12sleep20-21swapper/219:05:332
2182499279,4cyclictest0-21swapper/320:15:353
2182499275,5cyclictest7750irq/56-dwc_otg19:20:373
2182499272,5cyclictest7950irq/56-dwc_otg_20:30:273
2182499271,10cyclictest7950irq/56-dwc_otg_20:05:433
2182399271,17cyclictest0-21swapper/220:12:012
2182499265,14cyclictest26581-21expr20:50:413
2182399262,13cyclictest0-21swapper/221:10:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional