You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 09:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Aug 24, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
841999682,17cyclictest131rcu_preempt21:35:132
841999517,16cyclictest131rcu_preempt23:20:122
840299492,43cyclictest29655-21ssh22:05:140
841199483,9cyclictest131rcu_preempt20:30:121
840299484,36cyclictest5648-21ssh00:15:130
840299474,38cyclictest15334-21ssh22:15:130
842799442,26cyclictest35966irq/58-eth023:15:133
840299445,32cyclictest24347-21ssh22:20:140
8411994317,17cyclictest21-21ksoftirqd/123:15:111
841199409,23cyclictest21-21ksoftirqd/123:20:131
841999391,23cyclictest6243-21sshd20:15:142
8427993729,5cyclictest0-21swapper/323:25:113
8419993713,19cyclictest21472-21ssh23:30:152
841199374,21cyclictest28020-21ssh22:40:131
8402993720,12cyclictest12-21ksoftirqd/000:13:270
8402993720,12cyclictest12-21ksoftirqd/000:13:270
8411993612,16cyclictest24073-1kworker/u9:123:35:151
840299355,18cyclictest0-21swapper/022:45:140
841999342,20cyclictest25262-21ssh23:50:142
841199343,22cyclictest21-21ksoftirqd/123:30:151
840299342,22cyclictest14707-21ssh00:20:140
840299342,19cyclictest7171-21ssh23:40:110
8427993320,10cyclictest35966irq/58-eth022:50:123
8411993313,2cyclictest21-21ksoftirqd/100:05:141
840299334,22cyclictest12-21ksoftirqd/021:25:140
8427993215,10cyclictest0-21swapper/300:30:143
841199323,23cyclictest20607-21ssh22:00:131
8411993213,8cyclictest21-21ksoftirqd/123:05:131
840299321,5cyclictest0-21swapper/019:25:150
841999316,10cyclictest27-21ksoftirqd/220:45:142
841999314,17cyclictest131rcu_preempt00:25:152
841199312,22cyclictest11034-21sshd00:00:141
8411993116,11cyclictest0-21swapper/120:15:141
841199311,2cyclictest131rcu_preempt22:55:161
840299312,22cyclictest8553-21ssh00:34:360
841199300,23cyclictest0-21swapper/120:10:121
840299302,14cyclictest25880-21ssh21:45:140
841999292,21cyclictest2087-21df23:55:162
840299293,19cyclictest26577-21diskmemload23:00:150
8402992917,6cyclictest26577-21diskmemload21:10:130
842799285,10cyclictest35966irq/58-eth000:05:143
842799282,17cyclictest35966irq/58-eth021:25:133
842799281,17cyclictest35966irq/58-eth022:05:153
841999288,3cyclictest0-21swapper/223:15:122
841999283,17cyclictest27346irq/40-dwc2_hso21:50:122
8419992811,12cyclictest4697-21sshd22:45:152
841199282,20cyclictest1995-21ssh23:55:151
841199281,18cyclictest17359-21ssh22:50:241
842799273,19cyclictest35966irq/58-eth022:55:153
842799273,18cyclictest35966irq/58-eth020:55:123
842799271,16cyclictest35966irq/58-eth019:20:133
841999272,9cyclictest131rcu_preempt21:10:032
841999272,20cyclictest0-21swapper/200:00:152
841999271,7cyclictest131rcu_preempt23:25:122
8419992713,10cyclictest7413-21sh23:40:142
841999271,18cyclictest27346irq/40-dwc2_hso23:35:142
8411992715,6cyclictest0-21swapper/121:15:151
8411992714,7cyclictest1-21systemd22:45:141
840299273,16cyclictest3720-21ssh21:15:130
8402992715,8cyclictest12-21ksoftirqd/022:50:120
842799263,17cyclictest0-21swapper/323:55:163
842799262,18cyclictest35966irq/58-eth022:20:133
841999263,16cyclictest27346irq/40-dwc2_hso22:25:132
841999262,16cyclictest27346irq/40-dwc2_hso22:40:132
8411992611,9cyclictest8394-21cyclictest22:30:131
8411992611,9cyclictest8394-21cyclictest22:30:131
8402992611,10cyclictest12-21ksoftirqd/020:05:140
842799253,14cyclictest0-21swapper/323:45:123
8427992512,7cyclictest0-21swapper/319:25:143
841999257,7cyclictest131rcu_preempt20:50:282
841999256,9cyclictest23607-21cut22:00:222
841999256,12cyclictest27-21ksoftirqd/220:25:122
841999253,15cyclictest27346irq/40-dwc2_hso22:30:122
841999253,15cyclictest27346irq/40-dwc2_hso22:30:112
841999252,6cyclictest0-21swapper/200:39:342
841999251,16cyclictest27346irq/40-dwc2_hso22:35:122
841999250,18cyclictest27346irq/40-dwc2_hso19:20:132
841199252,18cyclictest12410-21ssh23:25:121
840299255,15cyclictest0-21swapper/000:05:150
8402992514,5cyclictest0-21swapper/023:45:120
8402992512,7cyclictest0-21swapper/021:37:030
842799249,11cyclictest0-21swapper/300:23:353
842799248,10cyclictest0-21swapper/300:39:353
842799242,18cyclictest35966irq/58-eth020:25:133
841999243,16cyclictest0-21swapper/200:24:352
8419992414,8cyclictest0-21swapper/223:45:152
841199243,4cyclictest131rcu_preempt23:50:131
8411992415,6cyclictest29120-21ssh00:10:151
8411992415,6cyclictest29120-21ssh00:10:141
8411992413,7cyclictest26577-21diskmemload00:25:161
840299249,7cyclictest0-21swapper/023:23:470
840299247,6cyclictest12-21ksoftirqd/021:31:060
840299244,5cyclictest12-21ksoftirqd/019:15:190
840299243,14cyclictest17592-21ssh23:10:140
840299242,16cyclictest11566-21ssh21:55:140
8402992410,6cyclictest12-21ksoftirqd/021:21:480
8402992410,5cyclictest0-21swapper/021:53:100
842799238,10cyclictest35966irq/58-eth023:35:153
842799232,15cyclictest35966irq/58-eth021:30:133
841999236,7cyclictest14313-21grep00:20:002
841999232,6cyclictest0-21swapper/200:13:352
841999232,6cyclictest0-21swapper/200:13:352
841999232,6cyclictest0-21swapper/200:09:342
841999232,2cyclictest131rcu_preempt22:50:092
841999232,15cyclictest12819-21ssh21:20:142
841999232,14cyclictest27346irq/40-dwc2_hso20:30:132
8419992316,3cyclictest27-21ksoftirqd/222:20:142
841199231,18cyclictest12364-21find20:55:121
841199231,17cyclictest7632-21ssh21:35:141
8411992310,7cyclictest8394-21cyclictest19:20:131
840299237,9cyclictest12-21ksoftirqd/022:00:130
840299236,3cyclictest131rcu_preempt22:25:110
840299236,11cyclictest12-21ksoftirqd/022:14:420
840299233,14cyclictest8509-21ssh23:05:140
8402992315,6cyclictest151rcuc/019:55:130
8402992313,7cyclictest32304-21cron00:30:010
8402992311,7cyclictest151rcuc/023:50:130
842799221,17cyclictest0-21swapper/322:45:143
841999229,9cyclictest8394-21cyclictest21:15:142
841999229,10cyclictest16532-21sshd23:09:152
841999225,11cyclictest0-21swapper/219:15:132
841999224,13cyclictest29724-21cpuspeed22:05:162
841999222,15cyclictest0-21swapper/200:30:352
841999221,2cyclictest131rcu_preempt21:00:112
841999221,2cyclictest131rcu_preempt21:00:112
841199222,4cyclictest131rcu_preempt22:20:131
841199222,15cyclictest844-21munin-node21:50:131
8411992210,9cyclictest2127-21cron21:50:011
840299228,8cyclictest12-21ksoftirqd/022:57:440
840299226,6cyclictest12-21ksoftirqd/000:00:390
840299225,7cyclictest12-21ksoftirqd/000:35:030
840299223,4cyclictest12-21ksoftirqd/000:00:010
840299223,3cyclictest131rcu_preempt22:40:160
8402992215,4cyclictest12-21ksoftirqd/022:40:010
8402992212,6cyclictest23692-21kthreadcore23:30:230
840299221,17cyclictest25723-21cut20:35:160
840299221,17cyclictest25723-21cut20:35:150
840299221,16cyclictest26771-21cut23:15:150
840299221,14cyclictest1760-21ssh23:35:250
8402992211,3cyclictest0-21swapper/023:26:010
8402992210,9cyclictest14423-21cron19:50:000
842799212,11cyclictest0-21swapper/300:10:153
842799212,11cyclictest0-21swapper/300:10:153
8427992111,6cyclictest35966irq/58-eth019:55:153
8427992110,7cyclictest0-21swapper/321:55:143
8427992110,6cyclictest35966irq/58-eth021:45:133
841999216,12cyclictest19787-21kthreadcore21:40:212
841999212,6cyclictest131rcu_preempt23:05:002
841999211,17cyclictest23005-21cut22:55:182
841199215,3cyclictest131rcu_preempt19:15:121
841199210,17cyclictest17232-21cpuspeed21:00:131
841199210,17cyclictest17232-21cpuspeed21:00:131
840299219,6cyclictest0-21swapper/020:45:140
8402992112,5cyclictest0-21swapper/019:20:250
840299211,16cyclictest7593-21ls20:50:160
842799203,9cyclictest0-21swapper/319:35:143
842799202,13cyclictest0-21swapper/321:10:123
8427992011,6cyclictest35966irq/58-eth023:05:133
8427992011,6cyclictest0-21swapper/323:40:123
842799201,12cyclictest0-21swapper/323:20:143
841999208,4cyclictest26388-21/usr/sbin/munin19:25:232
841999208,10cyclictest0-21swapper/223:13:422
841999206,12cyclictest27-21ksoftirqd/221:56:552
841999203,8cyclictest0-21swapper/219:55:192
841999201,9cyclictest26469-21kthreadcore20:00:202
841999201,11cyclictest22105-21fschecks_time21:25:162
841999201,10cyclictest7597-21kthreadcore22:10:202
841999201,10cyclictest135-21systemd-journal21:10:012
841999201,10cyclictest12813-21kthreadcore19:45:222
8419992010,8cyclictest0-21swapper/220:35:232
8419992010,8cyclictest0-21swapper/220:35:232
8419992010,3cyclictest15432-21sh22:15:152
841199209,8cyclictest14741-21df19:50:141
841199208,9cyclictest4958-21sed22:25:281
841199208,10cyclictest167-21systemd-network21:30:291
8411992010,2cyclictest21-21ksoftirqd/121:55:131
841199200,10cyclictest21492-21kworker/1:022:19:221
840299206,4cyclictest12-21ksoftirqd/020:30:150
840299205,3cyclictest131rcu_preempt19:10:210
840299203,5cyclictest19343-21latency_hist19:55:020
8402992013,4cyclictest0-21swapper/020:55:310
840299200,7cyclictest131rcu_preempt22:30:300
840299200,7cyclictest131rcu_preempt22:30:300
842799199,5cyclictest0-21swapper/321:40:213
842799198,7cyclictest35966irq/58-eth023:00:153
842799193,7cyclictest35966irq/58-eth020:45:123
842799192,6cyclictest0-21swapper/323:32:153
842799192,10cyclictest35966irq/58-eth000:00:143
8427991914,3cyclictest36066irq/59-eth000:15:143
842799191,13cyclictest0-21swapper/322:25:163
841999191,9cyclictest8726-21fschecks_count19:10:142
841999191,9cyclictest30457-21sshd20:40:152
841999191,9cyclictest29955-21kthreadcore19:30:222
841999191,9cyclictest29955-21kthreadcore19:30:222
841999191,9cyclictest1615-21fschecks_time20:10:152
841999191,8cyclictest2657766diskmemload21:45:152
841999191,8cyclictest14966-21kthreadcore20:55:222
841999191,8cyclictest1156-21kthreadcore19:35:202
841999191,15cyclictest5304-21grep19:40:172
841999191,15cyclictest11221-21fschecks_time20:20:162
841999191,10cyclictest32247-21ssh21:30:212
841999191,10cyclictest15012-21sed19:50:182
841199199,8cyclictest3790-21memory00:30:241
841199195,4cyclictest131rcu_preempt00:15:191
841199192,8cyclictest28695-21kthreadcore19:30:201
841199192,8cyclictest28695-21kthreadcore19:30:191
841199191,9cyclictest2191-21kthreadcore23:00:231
841199191,8cyclictest131rcu_preempt00:35:181
8411991914,3cyclictest21-21ksoftirqd/120:26:511
841199191,15cyclictest17304-21kthreadcore00:20:221
8411991910,6cyclictest25463-21cron21:45:011
840299196,3cyclictest131rcu_preempt19:40:160
840299194,3cyclictest131rcu_preempt20:40:290
840299194,3cyclictest131rcu_preempt19:30:230
840299194,3cyclictest131rcu_preempt19:30:230
840299191,9cyclictest17949-21kthreadcore20:25:210
840299191,7cyclictest12-21ksoftirqd/020:15:220
8402991913,3cyclictest0-21swapper/019:35:130
840299191,11cyclictest20443-21rm21:40:240
8402991910,6cyclictest12-21ksoftirqd/019:10:020
8402991910,5cyclictest26569-21latency_hist21:10:010
842799189,6cyclictest0-21swapper/322:41:293
842799188,6cyclictest0-21swapper/322:35:373
842799183,10cyclictest321rcuc/319:40:133
842799182,10cyclictest0-21swapper/320:10:133
842799182,10cyclictest0-21swapper/320:00:133
842799182,10cyclictest0-21swapper/300:25:053
842799181,9cyclictest0-21swapper/319:55:023
8427991812,4cyclictest0-21swapper/321:39:423
8427991810,6cyclictest35966irq/58-eth022:30:133
8427991810,6cyclictest35966irq/58-eth022:30:133
8427991810,6cyclictest0-21swapper/321:21:433
8427991810,4cyclictest0-21swapper/322:01:443
841999181,8cyclictest32444-21netstat20:05:242
841999181,14cyclictest8575-21latency_hist19:10:022
841199183,11cyclictest21909-21ssh21:25:141
841199181,9cyclictest16124-21/usr/sbin/munin23:45:291
8411991813,3cyclictest0-21swapper/122:35:111
8411991812,4cyclictest0-21swapper/119:45:191
841199181,10cyclictest19533-21diskstats19:55:171
8402991811,5cyclictest0-21swapper/020:20:130
840299181,13cyclictest1428-21apt-get20:10:120
8402991810,6cyclictest28810-21cron20:01:010
842799179,6cyclictest0-21swapper/321:50:053
842799179,3cyclictest0-21swapper/323:50:113
842799179,3cyclictest0-21swapper/322:12:163
842799177,7cyclictest0-21swapper/322:15:313
842799172,11cyclictest0-21swapper/319:10:123
8427991711,4cyclictest0-21swapper/321:15:283
8427991711,3cyclictest0-21swapper/323:10:133
841199178,7cyclictest11141-21users22:10:311
841199178,6cyclictest12800-21ssh21:20:131
841199176,3cyclictest131rcu_preempt20:40:221
841199175,9cyclictest24582-21sort20:00:161
841199173,12cyclictest32073-21tr22:05:231
841199171,3cyclictest0-21swapper/119:10:021
841199171,13cyclictest27368-21tr20:35:211
841199171,13cyclictest27368-21tr20:35:211
841199171,13cyclictest27182-21diskstats21:10:151
8411991711,3cyclictest131rcu_preempt23:10:131
841199171,12cyclictest8972-21ssh23:40:191
841199171,12cyclictest32512-21ntp_states20:05:271
841199171,12cyclictest11149-21sshd20:20:151
8411991710,5cyclictest13163-21sshd19:12:151
840299177,7cyclictest17358-21sshd21:00:150
840299177,7cyclictest17358-21sshd21:00:140
842799162,9cyclictest0-21swapper/320:35:053
842799162,9cyclictest0-21swapper/320:35:043
842799162,9cyclictest0-21swapper/320:20:133
842799161,12cyclictest0-21swapper/321:00:133
842799161,12cyclictest0-21swapper/321:00:133
842799161,10cyclictest0-21swapper/320:05:143
841199163,10cyclictest1431-21kthreadcore19:35:211
841199162,11cyclictest6247-21proc_pri20:45:271
841199161,12cyclictest24583-21cut21:05:211
841199161,12cyclictest23092-21apt-get19:25:131
841199161,11cyclictest5757-21grep19:40:181
841199161,11cyclictest11459-21grep20:50:301
842799151,7cyclictest0-21swapper/319:15:133
842799151,12cyclictest0-21swapper/319:10:013
842799151,11cyclictest0-21swapper/321:10:013
842799141,7cyclictest0-21swapper/319:30:123
842799141,7cyclictest0-21swapper/319:30:123
842799141,10cyclictest0-21swapper/320:40:223
842799141,10cyclictest0-21swapper/319:45:153
842799131,4cyclictest0-21swapper/320:30:193
842799122,6cyclictest0-21swapper/320:15:153
842799121,8cyclictest35966irq/58-eth020:50:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional