You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 13:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 01, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21151996423,19cyclictest21-21ksoftirqd/121:25:131
2116099608,5cyclictest131rcu_preempt20:35:142
2116099591,37cyclictest16262-21ssh22:45:102
2115199561,44cyclictest15088-21ssh00:35:121
2116899524,30cyclictest35966irq/58-eth023:05:133
2116099528,22cyclictest27-21ksoftirqd/222:10:132
2116099528,22cyclictest27-21ksoftirqd/222:10:122
21143995217,30cyclictest131rcu_preempt19:50:170
2115199519,4cyclictest131rcu_preempt22:15:141
2116099491,34cyclictest27177-21ssh22:15:132
2116099471,36cyclictest0-21swapper/200:15:112
21168994623,17cyclictest35966irq/58-eth021:25:123
2116099461,29cyclictest20513-21ssh00:20:172
2115199462,35cyclictest9188-21ssh22:05:121
2114399466,36cyclictest5399-21ssh23:15:140
2114399466,36cyclictest5399-21ssh23:15:130
21143994519,21cyclictest12-21ksoftirqd/022:50:130
21143994433,6cyclictest6843-21diskmemload22:25:110
2114399438,31cyclictest25349-21ssh21:20:130
21168994217,15cyclictest35966irq/58-eth023:20:123
2114399425,9cyclictest131rcu_preempt23:10:130
2114399421,36cyclictest7340-21ssh22:40:130
2114399412,31cyclictest23650-21ssh21:55:150
21160994019,15cyclictest27-21ksoftirqd/222:20:122
2114399401,32cyclictest10867-21ssh23:00:150
21160993916,13cyclictest21135-21cyclictest20:15:122
2114399394,27cyclictest7326-21ssh21:10:140
2115199381,8cyclictest131rcu_preempt00:25:131
2114399387,22cyclictest12-21ksoftirqd/019:20:150
2116099373,25cyclictest0-21swapper/222:51:572
2116099371,16cyclictest27090-21ssh23:45:132
2115199376,19cyclictest27071-21ssh23:45:131
21143993618,15cyclictest1829-21ssh22:55:130
2116099352,17cyclictest19305-21ssh22:27:332
21160993513,17cyclictest27-21ksoftirqd/221:35:112
21160993510,16cyclictest27-21ksoftirqd/220:25:132
21143993511,17cyclictest12-21ksoftirqd/021:40:130
2116899336,21cyclictest35966irq/58-eth023:55:143
2116899322,19cyclictest35966irq/58-eth022:55:133
2116099324,12cyclictest261rcuc/219:50:182
2115199323,22cyclictest18107-21ssh23:40:121
2115199321,25cyclictest19839-21sh23:05:141
2116899312,19cyclictest35966irq/58-eth023:00:143
2116099312,22cyclictest32498-21ssh23:30:142
21151993124,3cyclictest0-21swapper/121:30:121
2114399319,16cyclictest12-21ksoftirqd/019:10:120
2114399313,22cyclictest23468-21ssh23:25:110
2116899305,15cyclictest35966irq/58-eth022:10:143
2116899305,15cyclictest35966irq/58-eth022:10:143
2116899301,17cyclictest35966irq/58-eth022:40:123
2116099303,21cyclictest0-21swapper/219:40:142
2116099303,18cyclictest28850-21ssh23:10:152
21160993012,12cyclictest27-21ksoftirqd/200:30:142
2116899299,11cyclictest0-21swapper/300:00:113
2116099294,19cyclictest27346irq/40-dwc2_hso23:50:132
2116099292,21cyclictest32612-21sh00:07:432
2116099292,11cyclictest22034-21ssh21:53:332
2115199292,22cyclictest31286-21ssh22:51:561
2114399296,11cyclictest29456-21ssh00:25:140
2114399294,20cyclictest7489-21kworker/u8:000:15:120
2116099286,17cyclictest0-21swapper/221:40:152
2116099283,12cyclictest27346irq/40-dwc2_hso20:55:132
2116099282,19cyclictest0-21swapper/219:55:122
2116099281,15cyclictest7332-21ssh21:10:142
2116099281,15cyclictest32663-21ssh22:00:152
2114399282,5cyclictest131rcu_preempt19:25:140
2116099279,13cyclictest18110-21diskmemload23:40:132
2116099271,20cyclictest27346irq/40-dwc2_hso20:45:142
2115199272,19cyclictest844-21munin-node19:55:131
2115199272,19cyclictest15531-21ssh22:44:341
2115199272,18cyclictest30783-21ssh22:35:131
21151992719,4cyclictest0-21swapper/122:00:151
2115199271,18cyclictest23477-21ssh23:25:141
2114399273,14cyclictest12-21ksoftirqd/019:30:110
2114399272,9cyclictest0-21swapper/020:35:140
2114399272,19cyclictest3798-21ssh22:20:140
2114399272,19cyclictest27763-21ssh22:31:530
2114399272,19cyclictest14922-21ssh22:06:350
21143992713,6cyclictest0-21swapper/023:55:140
2116899263,15cyclictest0-21swapper/322:06:353
2116099269,12cyclictest27-21ksoftirqd/222:06:472
2116099267,9cyclictest27-21ksoftirqd/223:05:132
2116099261,7cyclictest9614-21cut20:05:172
21160992610,12cyclictest19434-21cron23:05:022
2114399269,9cyclictest131rcu_preempt20:55:130
2114399262,18cyclictest25557-21ssh22:13:340
2114399262,18cyclictest25557-21ssh22:13:330
2116899254,13cyclictest0-21swapper/321:10:143
2116899252,17cyclictest0-21swapper/323:35:153
2116899251,18cyclictest0-21swapper/322:01:453
2116099259,12cyclictest0-21swapper/220:00:292
2116099253,13cyclictest27346irq/40-dwc2_hso19:35:122
21160992512,11cyclictest27-21ksoftirqd/221:33:362
21160992511,9cyclictest6302-21switchtime00:10:252
2116099251,15cyclictest27511-21ls20:20:242
2115199252,17cyclictest6843-21diskmemload00:07:441
2115199251,18cyclictest2457-21ssh00:10:091
2114399257,15cyclictest12-21ksoftirqd/021:25:110
21143992511,11cyclictest20401-21ssh21:15:300
21143992510,11cyclictest2109-21ssh22:35:270
2116899242,13cyclictest35966irq/58-eth000:15:123
21168992410,9cyclictest35966irq/58-eth022:31:333
2116099249,11cyclictest27-21ksoftirqd/223:55:142
2116099249,11cyclictest20042-21proc_pri21:15:292
2116099242,17cyclictest0-21swapper/222:42:352
2116099242,13cyclictest25511-21/usr/sbin/munin22:30:262
21160992413,9cyclictest0-21swapper/223:15:142
21160992413,9cyclictest0-21swapper/223:15:132
21160992412,9cyclictest6870-21ssh22:39:562
2116099241,13cyclictest17129-21kthreadcore23:20:222
2116099241,12cyclictest25335-21apt21:20:122
21160992410,9cyclictest27-21ksoftirqd/200:25:282
2115199242,18cyclictest6071-21ssh00:30:141
2115199241,6cyclictest21795-21ntp_192.168.11422:10:241
2115199241,6cyclictest21795-21ntp_192.168.11422:10:231
2114399245,13cyclictest14403-21ssh23:20:130
2114399242,3cyclictest131rcu_preempt22:15:130
21143992414,7cyclictest2089-21cron00:10:010
21143992411,10cyclictest1493-21ssh22:00:200
2116899236,12cyclictest35966irq/58-eth023:15:153
2116899236,12cyclictest35966irq/58-eth023:15:143
2116899233,10cyclictest35966irq/58-eth022:35:133
2116899231,14cyclictest35966irq/58-eth020:35:133
2116099237,12cyclictest23619-21apt-get21:55:142
2116099235,14cyclictest30049-21fschecks_time21:00:172
2116099234,15cyclictest3860-21kthreadcore21:25:202
2116099232,16cyclictest0-21swapper/221:46:352
2116099231,6cyclictest999-21find20:30:122
2116099231,10cyclictest9445-21if_eth023:35:182
21151992312,5cyclictest6843-21diskmemload23:35:141
2115199231,17cyclictest25368-21ssh21:20:131
2114399233,16cyclictest6843-21diskmemload00:00:140
2114399233,11cyclictest0-21swapper/020:30:120
2114399231,7cyclictest131rcu_preempt21:52:330
21143992314,6cyclictest0-21swapper/023:35:020
21143992311,9cyclictest9510-21sed21:45:280
2116899223,16cyclictest0-21swapper/322:51:563
2116899223,13cyclictest35966irq/58-eth022:15:143
21168992212,7cyclictest35966irq/58-eth000:10:093
2116899221,13cyclictest0-21swapper/320:15:123
2116899221,11cyclictest35966irq/58-eth022:45:113
2116099227,4cyclictest27-21ksoftirqd/219:15:122
2116099223,9cyclictest21570-21/usr/sbin/munin00:00:142
2116099221,6cyclictest0-21swapper/219:30:052
2116099221,17cyclictest14268-21apt-get20:10:192
2116099221,15cyclictest27346irq/40-dwc2_hso20:40:122
2116099221,12cyclictest3552-21tr22:55:202
2116099221,11cyclictest26079-21cut23:25:212
2116099221,11cyclictest15798-21ssh00:35:192
2115199222,15cyclictest6843-21diskmemload22:28:341
21151992213,7cyclictest0-21swapper/119:40:241
21151992210,9cyclictest17576-21cut21:15:221
2114399221,16cyclictest15098-21ssh00:35:140
2116899219,9cyclictest0-21swapper/322:25:023
21168992112,4cyclictest0-21swapper/300:29:073
2116099213,15cyclictest22870-21hwlatdetect19:45:162
2116099211,9cyclictest7909-21apt-config19:30:012
2116099211,9cyclictest21449-21fschecks_count19:10:162
2116099211,15cyclictest20472-21sed20:50:162
2115199212,10cyclictest11103-21hwlatdetect23:00:161
2115199211,18cyclictest14629-21find21:50:131
2115199211,16cyclictest21761-21ssh22:30:121
2114399216,4cyclictest131rcu_preempt00:20:300
2114399213,15cyclictest19121-21timerandwakeup20:45:290
21143992112,6cyclictest19809-21sh23:05:120
21143992111,4cyclictest12-21ksoftirqd/019:45:150
2116899203,11cyclictest0-21swapper/322:27:343
2116899201,2cyclictest0-21swapper/323:45:133
2116099202,15cyclictest2360-21fschecks_count21:05:172
2116099201,9cyclictest21172-21latency_hist19:10:012
2115199208,3cyclictest131rcu_preempt19:35:011
2115199207,4cyclictest1007-21sendmail_mailtr21:00:271
21151992013,4cyclictest0-21swapper/122:45:101
2115199201,15cyclictest5526-21ssh23:15:171
2115199201,15cyclictest5526-21ssh23:15:171
2115199201,13cyclictest25191-21cut20:55:161
2115199201,12cyclictest5614-21ls21:05:251
21151992010,7cyclictest21-21ksoftirqd/120:30:121
2114399206,9cyclictest771-21cron20:30:010
2114399203,11cyclictest2436-21ssh00:10:090
2114399201,3cyclictest0-21swapper/023:41:010
2114399200,14cyclictest29885-21find21:00:120
2116899199,6cyclictest0-21swapper/323:50:563
2116899194,12cyclictest0-21swapper/321:40:213
2116899194,10cyclictest0-21swapper/323:25:143
2116899193,9cyclictest0-21swapper/321:55:343
2116899192,11cyclictest0-21swapper/321:35:133
2116899192,11cyclictest0-21swapper/320:30:133
2116899192,11cyclictest0-21swapper/300:30:133
2116899192,10cyclictest0-21swapper/321:00:143
2116899191,11cyclictest0-21swapper/320:55:023
21168991910,5cyclictest0-21swapper/321:50:343
2116099190,11cyclictest31341-21sed19:20:192
2115199192,14cyclictest28667-21apt-get20:25:141
2115199192,14cyclictest12602-21apt-get23:55:131
2115199191,9cyclictest7620-21switchtime22:20:271
2115199191,14cyclictest4951-21cut22:55:231
2115199191,12cyclictest5933-21ls21:45:161
2115199191,11cyclictest3379-21cut19:25:171
2114399198,8cyclictest3890-21ssh23:50:170
2114399192,11cyclictest21135-21cyclictest19:40:120
2114399191,6cyclictest4435-21munin-run20:00:010
21143991911,6cyclictest6182-21forks00:30:150
2114399191,14cyclictest23881-21apt-get20:20:080
2114399191,10cyclictest0-21swapper/022:47:320
2116899184,10cyclictest0-21swapper/323:30:143
2116899183,8cyclictest0-21swapper/319:45:143
2116899183,7cyclictest0-21swapper/323:40:123
2116899183,11cyclictest0-21swapper/300:08:183
2116899183,10cyclictest35966irq/58-eth021:20:143
2116899182,11cyclictest0-21swapper/321:47:343
2116899182,10cyclictest0-21swapper/319:35:053
2116899182,10cyclictest0-21swapper/319:30:123
2116899181,9cyclictest0-21swapper/321:30:113
2116899181,13cyclictest0-21swapper/319:55:013
2116899181,12cyclictest0-21swapper/300:20:313
2116899181,11cyclictest0-21swapper/300:35:293
2115199188,8cyclictest26420-21cat00:00:311
2115199186,9cyclictest9647-21grep20:35:291
2115199185,10cyclictest31022-21kthreadcore21:40:221
2115199184,4cyclictest22118-21kthreadcore00:20:241
2115199182,14cyclictest20567-21sed20:50:181
2115199182,10cyclictest14419-21ssh23:20:131
2115199181,9cyclictest17914-21wc20:10:341
2115199181,14cyclictest22128-21kthreadcore21:35:211
2115199181,13cyclictest32497-21diskmemload23:30:131
2115199180,7cyclictest10798-21memory21:10:231
2115199180,12cyclictest1824-21ls19:20:241
2114399189,7cyclictest12-21ksoftirqd/020:05:150
2114399188,7cyclictest2323-21df_inode21:05:150
21143991811,4cyclictest0-21swapper/020:00:230
21143991810,6cyclictest536-21usb-storage23:35:040
2114399180,16cyclictest18042-21ls20:10:340
2116899179,6cyclictest0-21swapper/323:10:063
2116899172,10cyclictest0-21swapper/319:40:133
2116899172,10cyclictest0-21swapper/319:10:133
2116899171,12cyclictest0-21swapper/320:40:283
2116899171,11cyclictest0-21swapper/320:55:233
2116899171,11cyclictest0-21swapper/320:25:173
2115199174,10cyclictest13888-21kthreadcore00:15:221
2115199173,11cyclictest25156-21tr20:20:161
2115199173,10cyclictest26075-21find19:15:121
2115199172,13cyclictest23700-21kthreadcore19:45:201
2115199172,11cyclictest27320-21ntp_states21:55:261
2115199172,10cyclictest0-21swapper/120:00:141
2115199172,10cyclictest0-21swapper/119:35:151
2115199171,9cyclictest10730-21cpuspeed20:40:141
2115199171,12cyclictest18693-21grep20:45:221
2114399177,8cyclictest20327-21df20:50:150
2114399175,9cyclictest10764-21cut20:40:150
2114399172,12cyclictest23707-21ntp_states21:35:260
2114399172,11cyclictest0-21swapper/019:15:140
21143991711,4cyclictest0-21swapper/023:45:120
2114399171,13cyclictest13645-21irqrtprio19:35:190
2114399171,12cyclictest11139-21diskstats21:30:150
21143991710,4cyclictest0-21swapper/020:15:240
2116899162,9cyclictest0-21swapper/321:05:153
2116899162,10cyclictest0-21swapper/319:55:133
2116899161,9cyclictest0-21swapper/321:15:153
2115199168,5cyclictest0-21swapper/119:10:271
2115199163,10cyclictest29383-21sed23:10:191
2115199162,11cyclictest3630-21ssh23:50:121
21151991611,3cyclictest19143-21cpuspeed20:15:141
2114399164,9cyclictest21219-21ls19:10:020
2116899154,5cyclictest0-21swapper/320:05:153
2116899151,7cyclictest0-21swapper/320:45:173
2115199151,4cyclictest0-21swapper/120:05:181
2115199151,11cyclictest27458-21apt-get19:50:181
2115199151,10cyclictest21255-21latency_hist19:10:021
2116899142,7cyclictest0-21swapper/319:15:123
2116899131,10cyclictest0-21swapper/319:25:203
2116899122,7cyclictest0-21swapper/320:25:003
2116899122,6cyclictest0-21swapper/319:20:153
2116899121,8cyclictest0-21swapper/320:00:183
2116899121,6cyclictest0-21swapper/320:10:223
2116899101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional