You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 15:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Fri Mar 13, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1507199674,45cyclictest21536-21ssh10:15:120
1508499585,38cyclictest3434-21ssh10:05:121
1507199584,45cyclictest14831-21ssh12:35:130
15098995718,24cyclictest0-21swapper/211:55:122
1507199564,7cyclictest131rcu_preempt08:25:130
1507199564,7cyclictest131rcu_preempt08:25:130
1507199553,45cyclictest5496-21ssh11:00:120
1507199545,7cyclictest131rcu_preempt08:55:120
1507199532,4cyclictest131rcu_preempt12:00:110
15071995232,17cyclictest12-21ksoftirqd/009:50:110
1508499513,37cyclictest31018-21ssh11:50:121
1509899495,33cyclictest12492-21ssh10:10:112
1509899461,31cyclictest14519-21ssh11:05:112
1509899461,31cyclictest14519-21ssh11:05:112
15071994624,10cyclictest12-21ksoftirqd/009:40:120
1511199453,27cyclictest28366irq/58-eth011:55:133
15111994527,11cyclictest28366irq/58-eth012:35:123
1509899451,25cyclictest2264-21ssh12:10:112
1509899451,25cyclictest2264-21ssh12:10:112
1511199444,25cyclictest28366irq/58-eth011:20:123
1511199442,26cyclictest28366irq/58-eth010:40:113
1509899433,17cyclictest10860-21ssh10:45:122
1507199431,4cyclictest131rcu_preempt07:40:120
1511199429,20cyclictest28366irq/58-eth007:40:133
1507199403,3cyclictest131rcu_preempt11:15:120
1508499398,19cyclictest17199-21kworker/u8:109:55:131
15071993925,3cyclictest12-21ksoftirqd/011:20:120
15098993829,5cyclictest0-21swapper/207:10:132
1508499372,25cyclictest32207-21ssh09:45:141
1507199361,23cyclictest20290-21ssh12:20:120
1511199356,23cyclictest28366irq/58-eth011:15:133
1511199354,19cyclictest28366irq/58-eth010:20:113
1509899351,28cyclictest9218-21cut12:30:222
1507199352,24cyclictest13254-21kthreadcore12:15:210
1507199351,27cyclictest25623-21sh12:05:120
15111993417,13cyclictest0-21swapper/310:50:113
15098993415,14cyclictest27-21ksoftirqd/208:45:122
15098993310,16cyclictest0-21swapper/211:48:132
1508499334,20cyclictest21977-21ssh11:45:151
1507199332,22cyclictest14538-21ssh11:05:130
1507199332,22cyclictest14538-21ssh11:05:130
1511199322,18cyclictest28366irq/58-eth009:10:113
1509899321,26cyclictest14262-21kthreadcore12:15:212
1508499321,22cyclictest7641-21ssh11:55:131
1507199322,22cyclictest4074-21ssh12:28:200
1511199315,18cyclictest28366irq/58-eth011:40:123
1509899311,15cyclictest3307-21sh11:34:482
15098993111,3cyclictest0-21swapper/209:30:132
1508499312,23cyclictest9449-21ssh11:36:221
1511199305,13cyclictest28366irq/58-eth009:25:123
1511199303,17cyclictest28366irq/58-eth011:10:123
1511199302,16cyclictest28366irq/58-eth012:20:103
1509899307,12cyclictest27-21ksoftirqd/211:20:502
1509899302,19cyclictest22346irq/40-dwc2_hso10:05:122
1508499305,5cyclictest21-21ksoftirqd/111:09:221
1508499305,5cyclictest21-21ksoftirqd/111:09:211
1507199302,20cyclictest10792-21ssh12:14:550
1507199302,20cyclictest10792-21ssh12:14:540
1509899297,15cyclictest27-21ksoftirqd/209:27:322
1509899293,16cyclictest22346irq/40-dwc2_hso07:55:112
1509899291,6cyclictest0-21swapper/212:28:202
1509899291,22cyclictest0-21swapper/211:19:442
1509899291,19cyclictest22346irq/40-dwc2_hso07:15:122
1509899291,11cyclictest6432-21cut11:35:212
15098992910,12cyclictest27-21ksoftirqd/209:20:052
1508499295,16cyclictest9348irq/52-mmc107:45:121
1508499293,21cyclictest0-21swapper/110:55:131
1507199292,19cyclictest6433-21tr11:35:210
1507199292,19cyclictest29964-21ssh11:49:200
15071992910,8cyclictest12-21ksoftirqd/007:50:110
15071992910,12cyclictest0-21swapper/010:40:120
1509899289,13cyclictest27-21ksoftirqd/211:54:172
1509899288,14cyclictest27-21ksoftirqd/208:55:292
1509899283,17cyclictest22346irq/40-dwc2_hso08:10:122
1509899283,16cyclictest12928-21ssh11:40:132
1509899282,20cyclictest0-21swapper/208:40:122
1509899282,19cyclictest0-21swapper/207:35:132
15098992810,12cyclictest27-21ksoftirqd/210:04:492
1509899280,13cyclictest27-21ksoftirqd/212:07:002
1508499283,17cyclictest12507-21ssh10:10:131
1507199282,19cyclictest16204-21grep11:40:210
15071992816,5cyclictest12-21ksoftirqd/010:30:140
15071992816,5cyclictest12-21ksoftirqd/010:30:140
1511199279,7cyclictest0-21swapper/309:30:113
1511199274,15cyclictest28366irq/58-eth010:00:113
1511199273,15cyclictest0-21swapper/311:35:213
1511199272,17cyclictest28366irq/58-eth010:45:133
15111992714,7cyclictest28366irq/58-eth010:15:113
1509899279,7cyclictest27-21ksoftirqd/210:51:142
1509899279,4cyclictest27-21ksoftirqd/211:29:172
1509899279,4cyclictest27-21ksoftirqd/211:29:172
1509899278,9cyclictest27-21ksoftirqd/209:55:192
1509899278,12cyclictest131rcu_preempt12:35:252
1509899276,7cyclictest27-21ksoftirqd/211:02:312
1509899275,3cyclictest131rcu_preempt10:18:092
1509899275,14cyclictest27-21ksoftirqd/210:59:022
1509899275,14cyclictest131rcu_preempt08:15:262
1509899273,16cyclictest22346irq/40-dwc2_hso08:00:102
15098992717,8cyclictest0-21swapper/207:25:132
1508499272,19cyclictest19811-21sh12:19:571
1508499271,15cyclictest0-21swapper/112:10:131
1508499271,15cyclictest0-21swapper/112:10:121
1507199274,17cyclictest25350-21ssh10:35:130
1511199267,9cyclictest28366irq/58-eth009:35:123
1511199263,17cyclictest0-21swapper/311:34:483
1511199263,15cyclictest0-21swapper/312:14:543
1511199263,15cyclictest0-21swapper/312:14:543
1511199262,16cyclictest28366irq/58-eth011:05:113
1511199262,16cyclictest28366irq/58-eth011:05:113
1509899269,4cyclictest27-21ksoftirqd/212:20:592
1509899268,12cyclictest27-21ksoftirqd/209:54:462
1509899268,11cyclictest27-21ksoftirqd/210:20:322
1509899267,4cyclictest27-21ksoftirqd/210:36:212
1509899266,8cyclictest27-21ksoftirqd/209:41:372
1509899266,5cyclictest27-21ksoftirqd/209:35:032
1509899265,7cyclictest27-21ksoftirqd/211:10:092
1509899264,3cyclictest131rcu_preempt07:40:282
1509899264,15cyclictest22346irq/40-dwc2_hso08:30:112
1509899262,3cyclictest131rcu_preempt07:50:192
1509899261,14cyclictest28585-21find09:05:112
1508499265,10cyclictest0-21swapper/108:45:131
1508499264,4cyclictest21-21ksoftirqd/110:45:151
15084992620,3cyclictest18957-21apt-get08:55:121
1507199262,18cyclictest7237-21ssh10:25:120
1511199252,17cyclictest0-21swapper/311:49:213
1511199251,17cyclictest28366irq/58-eth012:04:213
1509899258,5cyclictest27-21ksoftirqd/212:02:452
1509899258,11cyclictest27-21ksoftirqd/209:47:062
1509899256,5cyclictest27-21ksoftirqd/209:16:032
1509899256,13cyclictest27-21ksoftirqd/209:13:552
1509899252,14cyclictest27-21ksoftirqd/210:31:012
1509899252,14cyclictest27-21ksoftirqd/210:31:002
1509899251,15cyclictest21110-21ls08:20:212
1508499259,9cyclictest30589-21ssh10:20:111
1507199252,17cyclictest3308-21rm11:34:490
1507199252,14cyclictest10174-21ssh09:15:140
1509899246,12cyclictest27-21ksoftirqd/210:26:532
1509899242,14cyclictest22346irq/40-dwc2_hso08:35:102
1508499245,5cyclictest21-21ksoftirqd/111:23:061
1508499243,4cyclictest21-21ksoftirqd/112:35:011
15084992417,3cyclictest0-21swapper/109:25:141
1508499241,17cyclictest12944-21ssh11:40:131
1507199245,5cyclictest131rcu_preempt10:45:130
15071992419,2cyclictest0-21swapper/009:10:120
1507199241,16cyclictest14079-21diskmemload09:35:110
1509899231,8cyclictest27-21ksoftirqd/210:41:422
1508499236,13cyclictest0-21swapper/110:00:091
1508499235,5cyclictest21-21ksoftirqd/107:25:171
1508499234,4cyclictest21-21ksoftirqd/109:41:301
1508499233,5cyclictest21-21ksoftirqd/109:10:051
1508499233,4cyclictest21-21ksoftirqd/108:30:121
1508499232,5cyclictest21-21ksoftirqd/112:27:531
15084992318,3cyclictest201rcuc/111:02:191
15084992317,4cyclictest201rcuc/110:39:061
15084992312,7cyclictest24842-21apt-get07:20:101
1507199231,17cyclictest5859-21ssh12:30:130
1511199223,13cyclictest0-21swapper/312:30:213
1509899224,3cyclictest131rcu_preempt08:25:172
1509899224,3cyclictest131rcu_preempt08:25:162
1509899222,8cyclictest27-21ksoftirqd/208:05:262
1508499227,3cyclictest131rcu_preempt09:32:421
1508499225,5cyclictest21-21ksoftirqd/112:06:091
1508499223,5cyclictest21-21ksoftirqd/110:40:051
1508499223,4cyclictest21-21ksoftirqd/111:15:191
1508499223,3cyclictest21-21ksoftirqd/109:50:211
15084992217,3cyclictest201rcuc/109:19:111
15084992216,4cyclictest201rcuc/111:29:181
15084992216,4cyclictest201rcuc/111:29:171
1508499221,5cyclictest21-21ksoftirqd/110:50:191
1507199221,15cyclictest26803-21diskmemload10:00:110
1511199214,8cyclictest0-21swapper/307:50:113
1511199213,12cyclictest0-21swapper/310:05:133
1511199213,12cyclictest0-21swapper/309:15:143
15111992111,8cyclictest0-21swapper/311:53:013
1509899213,9cyclictest261rcuc/209:00:172
1508499215,5cyclictest21-21ksoftirqd/112:01:511
1508499214,13cyclictest0-21swapper/109:20:121
1508499213,6cyclictest21-21ksoftirqd/109:35:071
1508499213,5cyclictest21-21ksoftirqd/111:32:001
1508499213,4cyclictest21-21ksoftirqd/111:13:231
1508499212,5cyclictest21-21ksoftirqd/112:36:481
1508499211,6cyclictest21-21ksoftirqd/110:33:221
1508499211,6cyclictest21-21ksoftirqd/110:33:221
1508499211,6cyclictest21-21ksoftirqd/110:15:501
15084992116,3cyclictest201rcuc/112:23:051
15084992116,3cyclictest201rcuc/107:40:141
1508499211,4cyclictest21-21ksoftirqd/110:27:061
1508499211,4cyclictest21-21ksoftirqd/108:30:021
1508499211,4cyclictest21-21ksoftirqd/108:30:011
15084992112,5cyclictest14167-21apt-get08:50:131
1511199203,12cyclictest0-21swapper/312:29:593
15111992012,5cyclictest28366irq/58-eth009:20:133
1511199201,13cyclictest0-21swapper/308:45:023
15111992010,6cyclictest0-21swapper/310:14:163
1509899203,12cyclictest27-21ksoftirqd/207:45:102
1509899201,9cyclictest17662-21grep08:50:262
1508499207,9cyclictest21235-21apt-get07:50:121
1508499202,4cyclictest21-21ksoftirqd/108:00:271
1508499202,4cyclictest21-21ksoftirqd/107:10:291
1508499201,5cyclictest21-21ksoftirqd/108:35:101
15084992015,3cyclictest28594-21apt-get09:05:111
1508499201,13cyclictest27439-21ls09:00:281
1507199209,8cyclictest8032-21apt-get08:10:120
1507199208,3cyclictest131rcu_preempt11:55:300
1507199207,9cyclictest16427-21apt-get07:45:120
1507199203,4cyclictest12-21ksoftirqd/009:46:430
1507199202,15cyclictest27339-21apt-get08:30:100
1507199202,14cyclictest899-21grep09:10:020
1507199202,14cyclictest23537-21ssh11:10:140
1507199202,14cyclictest16133-21ssh10:10:230
1507199200,13cyclictest24592-21latency_hist07:20:010
1511199199,6cyclictest0-21swapper/310:28:273
1511199197,8cyclictest0-21swapper/309:55:143
1511199193,11cyclictest0-21swapper/312:19:213
1511199193,11cyclictest0-21swapper/312:05:213
1511199191,13cyclictest0-21swapper/309:50:023
1511199191,13cyclictest0-21swapper/308:00:123
1511199191,12cyclictest0-21swapper/308:25:013
1509899191,15cyclictest0-21swapper/207:30:132
1509899191,10cyclictest25332-21irqcore07:20:162
1508499191,5cyclictest21-21ksoftirqd/108:05:261
1508499191,4cyclictest21-21ksoftirqd/108:20:011
1508499191,10cyclictest4533-21/usr/sbin/munin08:40:281
1507199195,9cyclictest12-21ksoftirqd/010:50:110
1507199192,13cyclictest27947-21apt-get09:25:020
1507199191,9cyclictest14610-21kthreadcore08:15:210
1507199191,13cyclictest18791-21apt-get09:20:040
15071991910,6cyclictest1796-21cron07:30:000
1511199189,6cyclictest0-21swapper/310:38:083
1511199189,6cyclictest0-21swapper/310:33:333
1511199189,6cyclictest0-21swapper/310:33:323
1511199188,8cyclictest0-21swapper/311:28:403
1511199188,8cyclictest0-21swapper/311:28:403
1511199182,10cyclictest0-21swapper/309:00:153
1511199182,10cyclictest0-21swapper/308:30:113
1511199182,10cyclictest0-21swapper/308:15:133
1511199182,10cyclictest0-21swapper/307:45:143
1511199182,10cyclictest0-21swapper/307:30:123
1511199182,10cyclictest0-21swapper/307:10:123
15111991810,6cyclictest0-21swapper/310:56:453
1508499189,6cyclictest6569-21sshd07:34:221
1508499185,7cyclictest0-21swapper/108:10:311
1508499183,4cyclictest21-21ksoftirqd/107:55:201
1508499183,4cyclictest21-21ksoftirqd/107:15:121
1508499183,12cyclictest11370-21basename07:40:011
1507199186,8cyclictest12-21ksoftirqd/008:40:130
1507199185,2cyclictest131rcu_preempt11:54:310
1507199183,4cyclictest131rcu_preempt11:25:130
1507199183,4cyclictest131rcu_preempt11:25:120
1507199182,12cyclictest30711-21diskstats10:20:140
1507199182,11cyclictest21484-21ntp_states09:55:260
1507199181,7cyclictest7183-21ntp_states10:05:260
1507199181,7cyclictest5090-21kthreadcore08:05:200
1507199181,13cyclictest30661-21latency_hist08:00:020
1507199181,11cyclictest5129-21cpuspeed09:30:130
1507199180,12cyclictest403-21ls10:55:300
1511199172,10cyclictest0-21swapper/309:40:123
1511199172,10cyclictest0-21swapper/308:35:113
15111991710,5cyclictest0-21swapper/311:00:213
15111991710,5cyclictest0-21swapper/307:19:473
1508499175,2cyclictest131rcu_preempt08:20:111
1507199177,8cyclictest2120-21df_inode07:30:130
1507199174,10cyclictest29412-21cron07:25:010
1507199171,5cyclictest131rcu_preempt08:45:120
15071991712,3cyclictest3002-21latency08:35:200
1507199171,13cyclictest8883-21kthreadcore07:35:200
1507199171,11cyclictest21401-21switchtime08:20:250
1507199170,11cyclictest18951-21ls07:10:260
1511199162,10cyclictest0-21swapper/307:20:113
1511199161,7cyclictest0-21swapper/309:50:193
1511199161,10cyclictest0-21swapper/308:10:133
1507199166,7cyclictest30871-21apt-get08:00:100
15071991611,3cyclictest18372-21usage_EthcTask08:50:310
1507199161,12cyclictest26969-21tr09:00:230
1511199151,12cyclictest0-21swapper/308:55:023
1511199151,12cyclictest0-21swapper/308:25:173
1511199151,12cyclictest0-21swapper/308:25:173
1511199141,7cyclictest0-21swapper/308:10:013
1509899142,8cyclictest15176-21latency_hist07:10:022
1508499142,9cyclictest15142-21latency_hist07:10:031
1511199138,3cyclictest0-21swapper/308:59:283
1511199133,8cyclictest0-21swapper/309:05:273
1511199132,9cyclictest0-21swapper/307:35:143
1511199131,8cyclictest0-21swapper/308:45:143
1507199135,5cyclictest15159-21latency_hist07:10:020
1511199121,9cyclictest33-21ksoftirqd/307:25:153
1511199111,7cyclictest0-21swapper/307:55:193
1511199101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional