You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 10:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 08, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1230022590,2sleep10-21swapper/104:25:591
393521620,5sleep01361099cyclictest08:40:560
3189821570,5sleep21361299cyclictest08:35:462
136109910427,60cyclictest61-21kswapd008:20:510
78172950,5sleep01361099cyclictest07:27:040
1361099916,45cyclictest0-21swapper/008:30:540
134462910,2sleep30-21swapper/307:34:443
134462910,2sleep30-21swapper/307:34:443
255202870,5sleep01361099cyclictest08:27:190
13610998318,15cyclictest0-21swapper/007:24:200
1361099831,28cyclictest13940-21fschecks_time04:30:570
13610998027,6cyclictest30215-21aten_rbpower_po07:15:520
13610998010,15cyclictest0-21swapper/005:35:530
13610997963,3cyclictest0-21swapper/007:05:500
13610997933,6cyclictest0-21swapper/006:45:520
13610997818,17cyclictest32427-21ssh06:39:440
275072760,3sleep229-21rcuc/207:51:012
1361099747,11cyclictest0-21swapper/005:50:520
1361099747,11cyclictest0-21swapper/005:50:520
13610997432,3cyclictest0-21swapper/006:40:530
13610997423,4cyclictest7850irq/56-dwc_otg_09:05:590
13610997420,4cyclictest6752-21sed08:05:400
1361099741,26cyclictest27910-21cat05:10:400
188712730,15sleep17950irq/56-dwc_otg_07:01:081
188712730,15sleep17950irq/56-dwc_otg_07:01:081
140582730,3sleep2301ktimersoftd/208:14:042
1361099738,16cyclictest0-21swapper/009:14:230
1361099731,20cyclictest10086-21taskset08:09:230
1361099731,20cyclictest10086-21taskset08:09:230
1361099721,23cyclictest908-21runrttasks09:37:000
287332710,3sleep01361099cyclictest09:49:490
1361099713,26cyclictest0-21swapper/009:50:570
13610997122,39cyclictest0-21swapper/006:25:510
13610997116,43cyclictest0-21swapper/006:04:170
1361099711,25cyclictest11621-21ssh08:50:390
1361099711,21cyclictest11710-21grep08:10:590
13610997027,4cyclictest7750irq/56-dwc_otg06:50:510
13610997018,3cyclictest0-21swapper/005:16:000
1361099695,14cyclictest7850irq/56-dwc_otg_08:50:500
13610996922,7cyclictest908-21runrttasks09:25:460
1361099692,17cyclictest0-21swapper/007:13:240
1361099691,25cyclictest0-21swapper/005:05:400
1361099691,15cyclictest0-21swapper/009:41:010
285362680,2sleep31361399cyclictest09:49:283
1361099682,21cyclictest0-21swapper/009:00:410
13610996815,4cyclictest30663-21apt-config07:55:400
312352670,2sleep10-21swapper/109:13:221
14492670,2sleep30-21swapper/308:37:163
13611996724,19cyclictest16384-21/usr/sbin/munin04:36:061
13611996724,19cyclictest16384-21/usr/sbin/munin04:36:061
1361099671,19cyclictest0-21swapper/008:17:550
65032660,2sleep00-21swapper/009:22:190
3832660,2sleep30-21swapper/306:40:283
13611996521,19cyclictest27730-21sh09:47:591
1361099658,19cyclictest0-21swapper/009:20:060
1361099652,19cyclictest0-21swapper/009:55:520
1361099651,7cyclictest19831-21chrt07:41:130
1361099651,5cyclictest24366-21apt-config09:05:410
13610996515,10cyclictest908-21runrttasks07:32:480
13610996515,10cyclictest908-21runrttasks07:32:480
13610996512,14cyclictest21302-21fschecks_count06:20:560
1361099651,19cyclictest25577-21systemd-journal04:36:160
1361099651,19cyclictest25577-21systemd-journal04:36:160
1361099651,18cyclictest0-21swapper/009:31:030
13611996422,10cyclictest0-21swapper/108:00:511
1361099643,29cyclictest31003-21ssh07:55:500
1361099642,17cyclictest0-21swapper/007:35:560
13610996415,9cyclictest0-21swapper/007:04:430
13610996415,9cyclictest0-21swapper/007:04:430
1361099633,15cyclictest0-21swapper/006:15:310
1361099632,21cyclictest0-21swapper/007:46:520
78572620,1sleep20-21swapper/208:45:532
42602620,2sleep30-21swapper/306:44:433
13612996219,16cyclictest7850irq/56-dwc_otg_08:50:502
1361099626,15cyclictest0-21swapper/006:55:460
1361099621,21cyclictest0-21swapper/004:44:080
13611996121,25cyclictest24783-21ssh09:45:111
1361099612,18cyclictest0-21swapper/005:13:380
1361099611,6cyclictest32691-21grep08:36:020
13611996010,23cyclictest18491-21ssh07:40:531
13610996014,3cyclictest0-21swapper/005:35:390
1361099601,22cyclictest0-21swapper/004:49:260
13611995925,20cyclictest953450rm07:30:101
13611995922,19cyclictest8617-21ssh08:06:431
13611995922,19cyclictest8617-21ssh08:06:431
1361099592,32cyclictest8572-21fschecks_time05:45:580
13610995916,8cyclictest0-21swapper/004:52:050
1361099591,32cyclictest2175-21unixbench_multi05:26:070
1361099591,32cyclictest2175-21unixbench_multi05:26:070
1361099591,18cyclictest0-21swapper/006:31:280
13611995820,24cyclictest3684-21rm08:40:511
13611995820,22cyclictest7850irq/56-dwc_otg_07:12:591
13611995819,20cyclictest3243-21ssh08:40:291
13611995816,27cyclictest5074-21sh07:24:371
1361199574,4cyclictest0-21swapper/108:10:501
13611995726,21cyclictest10872-21/usr/sbin/munin05:51:031
13611995726,21cyclictest10872-21/usr/sbin/munin05:51:031
13611995724,18cyclictest12699-21/usr/sbin/munin05:56:021
13611995723,12cyclictest10142-21/usr/sbin/munin06:51:031
13611995716,18cyclictest25134-21ssh08:26:351
13610995711,17cyclictest0-21swapper/005:00:020
203902560,6sleep21361299cyclictest07:03:432
203902560,6sleep21361299cyclictest07:03:432
1361199569,37cyclictest0-21swapper/108:50:501
1361199565,27cyclictest7850irq/56-dwc_otg_09:15:521
13611995622,20cyclictest8026-21/usr/sbin/munin08:45:561
13611995618,21cyclictest8344-21ssh09:25:391
13611995617,22cyclictest15873-21ssh09:34:041
13611995615,31cyclictest7750irq/56-dwc_otg08:21:051
13611995611,14cyclictest19080-21apt08:20:391
1361099561,31cyclictest15735-21aten_rbpower_vo06:05:540
13611995525,20cyclictest24390-21diskmemload08:30:511
13611995522,14cyclictest9650-21taskset05:48:151
13611995521,7cyclictest7850irq/56-dwc_otg_07:15:491
13611995519,12cyclictest30004-21aten_rbpower_vo05:15:551
13611995517,13cyclictest30413-21chrt07:55:231
13611995516,26cyclictest21049-21cut06:20:531
13611995516,19cyclictest30698-21ssh06:36:451
13611995516,13cyclictest13821-21ssh07:35:241
13611995516,13cyclictest13821-21ssh07:35:241
1361099552,28cyclictest0-21swapper/005:41:080
13611995424,8cyclictest28290-21df05:10:541
13611995422,13cyclictest23757-21ssh07:07:171
13611995421,9cyclictest29536-21apt-get09:50:521
221992530,2sleep20-21swapper/206:21:072
13611995326,13cyclictest23040-21/usr/sbin/munin06:25:551
13611995324,12cyclictest22807-21ssh09:03:211
13611995317,21cyclictest26362-21ssh06:31:391
13611995317,13cyclictest4940-21rm10:00:271
13611995316,25cyclictest11587-21sh09:28:491
13611995316,11cyclictest24442-21ssh09:05:431
251902520,1sleep20-21swapper/209:45:402
13611995226,12cyclictest18982-21ssh09:37:111
13611995216,25cyclictest4740-21ssh06:45:331
13611995136,12cyclictest1048-21apt-get05:25:501
13611995136,12cyclictest1048-21apt-get05:25:501
13611995121,13cyclictest14764-21/usr/sbin/munin04:31:081
13611995118,21cyclictest18355-21ssh08:57:471
1361199505,6cyclictest0-21swapper/105:40:521
1361199504,34cyclictest0-21swapper/107:55:511
13611995024,11cyclictest14885-21ssh06:56:161
13611995022,18cyclictest7850irq/56-dwc_otg_07:35:551
13611995022,13cyclictest6479-21users06:46:091
293592490,2sleep31361399cyclictest07:14:553
13611994914,17cyclictest25971-21ssh07:49:491
13611994914,12cyclictest24173-21taskset04:59:571
13611994913,15cyclictest18705-21runrttasks04:44:181
1361299487,28cyclictest0-21swapper/206:50:532
13612994826,14cyclictest7950irq/56-dwc_otg_07:45:522
13611994816,14cyclictest15260-21taskset06:04:261
1361099482,6cyclictest0-21swapper/005:20:520
13613994726,4cyclictest7950irq/56-dwc_otg_07:25:513
132152472,14sleep00-21swapper/004:29:030
1361299467,24cyclictest0-21swapper/206:05:532
13611994622,13cyclictest18945-21sshd06:15:221
1361399457,25cyclictest0-21swapper/306:20:523
1361299457,24cyclictest0-21swapper/205:05:532
13611994419,18cyclictest16895-21taskset06:07:121
13611994416,18cyclictest5977-21seq05:37:381
13611994412,21cyclictest0-21swapper/105:00:531
13611994412,18cyclictest20541-21chrt06:17:481
13611994321,4cyclictest22442-21idleruntime-cro04:55:391
1361199432,10cyclictest0-21swapper/105:30:531
13611994310,14cyclictest20514-21taskset04:49:211
1361399427,12cyclictest0-21swapper/309:00:403
1361299427,22cyclictest0-21swapper/208:55:542
13611994231,7cyclictest221ktimersoftd/105:20:511
13611994210,9cyclictest0-21swapper/105:05:511
1361099426,23cyclictest0-21swapper/006:15:550
13613994034,4cyclictest27810-21sh08:30:513
1361299405,21cyclictest29026-21ssh06:35:522
88642390,1sleep30-21swapper/306:50:273
245472390,9sleep3381ktimersoftd/305:00:543
13613993918,5cyclictest39-21ksoftirqd/308:00:403
1361299397,23cyclictest0-21swapper/208:40:532
13612993918,4cyclictest29790-21ssh09:50:562
13610993915,15cyclictest0-21swapper/005:56:110
1361399386,21cyclictest0-21swapper/305:05:533
1361399381,23cyclictest21197-21cut04:50:583
13612993816,9cyclictest29-21rcuc/209:50:402
13612993810,6cyclictest0-21swapper/209:08:192
1361399378,5cyclictest39-21ksoftirqd/306:50:583
1361399377,24cyclictest0-21swapper/308:10:533
1361299378,14cyclictest7950irq/56-dwc_otg_06:56:052
1361299377,5cyclictest0-21swapper/209:16:042
13612993721,4cyclictest4532-21/usr/sbin/munin09:20:512
1361399369,18cyclictest25609-21ssh07:10:393
1361399364,14cyclictest7950irq/56-dwc_otg_06:16:013
1361299366,5cyclictest7750irq/56-dwc_otg07:16:062
13612993613,6cyclictest7950irq/56-dwc_otg_07:55:532
1361399358,14cyclictest7950irq/56-dwc_otg_08:21:063
1361399357,24cyclictest0-21swapper/305:45:543
1361399351,25cyclictest11636-21latency_hist08:50:403
1361299356,6cyclictest7850irq/56-dwc_otg_09:05:412
13612993515,5cyclictest13463-21sh09:30:592
13612993511,3cyclictest7750irq/56-dwc_otg07:10:522
1361399345,14cyclictest7950irq/56-dwc_otg_05:40:573
1361399343,14cyclictest7950irq/56-dwc_otg_09:06:333
13613993413,6cyclictest39-21ksoftirqd/305:21:053
1361399341,23cyclictest1311-21cut05:25:563
1361399341,23cyclictest1311-21cut05:25:563
13613993411,13cyclictest0-21swapper/309:11:043
13613993410,6cyclictest7950irq/56-dwc_otg_06:11:083
1361299348,14cyclictest14336-21aten_rbpower_cu07:35:522
1361299346,6cyclictest7950irq/56-dwc_otg_08:05:412
1361299341,5cyclictest7950irq/56-dwc_otg_04:55:522
13612993410,14cyclictest7743-21grep08:06:022
13612993410,14cyclictest7743-21grep08:06:022
1361399337,19cyclictest37-21rcuc/305:10:523
1361399337,14cyclictest7950irq/56-dwc_otg_07:51:083
1361399336,14cyclictest7950irq/56-dwc_otg_08:51:033
1361399334,5cyclictest18279-21latency_hist07:40:403
1361399333,14cyclictest7950irq/56-dwc_otg_09:40:393
1361399332,20cyclictest18961-21ssh07:01:093
1361399332,20cyclictest18961-21ssh07:01:093
1361399331,22cyclictest7568-21ls08:06:003
1361399331,22cyclictest7568-21ls08:06:003
13613993310,14cyclictest0-21swapper/307:45:523
1361299339,12cyclictest8637-21apt-key09:25:412
1361299338,5cyclictest7950irq/56-dwc_otg_06:15:582
1361299338,14cyclictest7950irq/56-dwc_otg_06:10:512
1361299337,12cyclictest7950irq/56-dwc_otg_08:15:562
1361299336,13cyclictest7950irq/56-dwc_otg_07:40:572
1361299332,6cyclictest0-21swapper/207:25:402
13612993319,7cyclictest22712chrt05:26:212
13612993319,7cyclictest22712chrt05:26:212
1361299331,7cyclictest20039-21chrt09:39:042
13612993313,5cyclictest31-21ksoftirqd/205:15:402
1361299331,23cyclictest0-21swapper/206:46:072
1361299331,21cyclictest25047-21cut06:30:572
1361399328,11cyclictest0-21swapper/309:55:543
1361399327,16cyclictest0-21swapper/304:30:533
1361399327,13cyclictest7950irq/56-dwc_otg_09:15:543
1361399326,14cyclictest7950irq/56-dwc_otg_07:19:353
1361399325,13cyclictest7950irq/56-dwc_otg_08:01:013
1361399324,12cyclictest7950irq/56-dwc_otg_05:51:083
1361399324,12cyclictest7950irq/56-dwc_otg_05:51:083
1361399322,9cyclictest7950irq/56-dwc_otg_08:15:513
13613993213,6cyclictest39-21ksoftirqd/309:05:403
1361399321,27cyclictest17054-21ls04:40:393
1361399321,27cyclictest17054-21ls04:40:393
13613993212,5cyclictest0-21swapper/307:00:403
1361399321,23cyclictest12764-21ls09:30:413
1361399321,22cyclictest30572-21timerandwakeup09:51:063
1361299327,15cyclictest0-21swapper/204:30:532
1361299322,4cyclictest7750irq/56-dwc_otg09:10:522
13612993217,5cyclictest31-21ksoftirqd/205:55:522
13612993216,4cyclictest1728-21processes06:41:042
1361299321,3cyclictest7850irq/56-dwc_otg_08:31:392
13612993212,5cyclictest6470-21/usr/sbin/munin05:41:072
1361299321,21cyclictest9969-21ls07:30:412
1361299321,11cyclictest7950irq/56-dwc_otg_08:26:092
1361399317,6cyclictest7950irq/56-dwc_otg_09:31:083
1361399316,15cyclictest0-21swapper/307:42:313
1361399315,14cyclictest7950irq/56-dwc_otg_04:55:533
1361399313,13cyclictest3787-21wget08:40:543
1361399312,19cyclictest0-21swapper/309:23:153
1361399311,8cyclictest7950irq/56-dwc_otg_05:15:553
13613993116,6cyclictest39-21ksoftirqd/308:26:093
1361399311,23cyclictest18205-21timerandwakeup04:41:063
1361299319,5cyclictest0-21swapper/207:05:572
1361299317,22cyclictest0-21swapper/204:45:542
1361299317,15cyclictest0-21swapper/205:50:532
1361299317,15cyclictest0-21swapper/205:50:532
1361299313,18cyclictest0-21swapper/209:56:002
1361299311,20cyclictest0-21swapper/205:15:582
1361299311,13cyclictest7950irq/56-dwc_otg_06:01:072
1361399304,11cyclictest7750irq/56-dwc_otg05:55:573
1361399301,3cyclictest0-21swapper/306:31:253
1361399301,22cyclictest6311-21ls05:40:403
13612993015,4cyclictest5114-21/usr/sbin/munin05:35:592
13612993013,5cyclictest9348-21unixbench-2d05:46:082
1361299301,23cyclictest24999-21latency05:01:002
13612993011,6cyclictest7850irq/56-dwc_otg_07:35:402
13612993011,6cyclictest7850irq/56-dwc_otg_07:35:402
13612993010,12cyclictest0-21swapper/208:21:002
126882300,1sleep30-21swapper/304:26:053
1361399292,8cyclictest39-21ksoftirqd/309:40:583
1361399292,13cyclictest15542-21latency_hist06:05:403
1361399291,21cyclictest2375-21ssh07:20:593
1361299298,5cyclictest7950irq/56-dwc_otg_06:30:402
1361299297,6cyclictest7950irq/56-dwc_otg_05:21:022
1361299291,19cyclictest29-21rcuc/204:35:582
1361299291,19cyclictest29-21rcuc/204:35:582
13613992819,6cyclictest20635-21idleruntime-cro04:50:393
13612992815,4cyclictest17779-21latency04:41:002
1361299281,18cyclictest31-21ksoftirqd/204:51:532
1361399272,5cyclictest7950irq/56-dwc_otg_06:25:483
13612992719,5cyclictest3961-21unixbench_multi05:31:072
1361399266,9cyclictest7950irq/56-dwc_otg_05:30:533
116372263,12sleep20-21swapper/204:25:412
1361399247,14cyclictest0-21swapper/306:05:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional