You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 09:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Jun 28, 2026 00:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1216399665,49cyclictest31410-21ssh00:10:150
12163996624,27cyclictest15037-21ssh23:25:130
1216399583,11cyclictest131rcu_preempt21:40:130
1216399562,42cyclictest23727-21ssh22:00:130
12252995534,12cyclictest35966irq/58-eth000:35:123
1218199524,38cyclictest30634-21ssh21:10:121
1218199512,40cyclictest25728-21ssh22:55:151
1218199473,39cyclictest10920-21ssh21:35:131
12181994617,20cyclictest7977-21ssh00:15:141
1216399461,36cyclictest16305-21ssh21:20:140
1225299454,28cyclictest35966irq/58-eth021:25:153
1218199456,29cyclictest300-21ssh22:05:141
1218199456,29cyclictest300-21ssh22:05:141
1222099441,28cyclictest2370-21ssh23:00:142
1225299431,28cyclictest35966irq/58-eth021:15:153
1225299414,22cyclictest35966irq/58-eth021:10:123
1225299406,18cyclictest35966irq/58-eth023:55:133
12181994024,12cyclictest23186-21apt-get19:55:131
1222099393,28cyclictest27346irq/40-dwc2_hso21:55:132
1222099393,28cyclictest27346irq/40-dwc2_hso21:55:132
12181993923,11cyclictest31399-21apt-get00:10:141
1216399396,23cyclictest30252-21diskmemload00:20:130
1218199382,25cyclictest0-21swapper/123:00:131
1218199381,29cyclictest14549-21ssh21:55:121
1218199381,29cyclictest14549-21ssh21:55:121
1218199379,11cyclictest21298-21ssh23:27:091
12181993728,4cyclictest23708-21apt-get22:00:121
12181993725,6cyclictest19925-21apt-get21:40:131
12181993724,7cyclictest18379-21apt-get19:50:111
12181993711,11cyclictest11505-21ssh22:28:321
1216399372,25cyclictest9396-21ssh22:10:130
1225299364,18cyclictest35966irq/58-eth022:30:113
1218199368,11cyclictest7882-21diskstats22:45:151
1218199368,11cyclictest7882-21diskstats22:45:151
12181993612,10cyclictest7299-21ssh22:44:561
1216399363,21cyclictest22165-21ssh22:35:130
1225299354,19cyclictest0-21swapper/323:00:133
1218199359,11cyclictest30210-21cut23:50:211
12181993527,3cyclictest31604-21apt-get19:30:111
12181993521,10cyclictest16293-21apt-get21:20:141
1218199351,30cyclictest1907-21ssh21:30:121
12181993511,10cyclictest21809-21rm00:20:341
12181993511,10cyclictest14540-21ssh22:10:501
12181993510,10cyclictest21434-21ssh00:04:271
12181993510,10cyclictest21434-21ssh00:04:271
12163993516,7cyclictest131rcu_preempt20:55:120
1222099340,18cyclictest24418-21ssh23:46:352
1218199349,10cyclictest8921-21ssh00:32:271
12181993426,3cyclictest16026-21apt-get20:55:121
12181993423,7cyclictest25664-21apt-get21:05:131
12181993413,7cyclictest19184-21ssh22:31:561
12181993410,11cyclictest8362-21ssh23:38:511
12181993410,11cyclictest8362-21ssh23:38:511
12181993312,10cyclictest16951-21ssh23:06:251
12181993311,10cyclictest26277-21ssh23:11:451
12181993311,10cyclictest26277-21ssh23:11:451
12252993214,15cyclictest0-21swapper/323:50:133
1222099326,12cyclictest15053-21diskmemload23:25:142
1218199329,9cyclictest23515-21grep20:25:281
1218199329,7cyclictest30252-21diskmemload23:32:051
1218199328,11cyclictest30344-21ssh21:25:401
12181993223,3cyclictest14843-21apt-get20:20:141
12181993220,8cyclictest0-21swapper/123:20:121
12181993211,7cyclictest30252-21diskmemload21:18:521
12181993210,8cyclictest30252-21diskmemload00:06:551
12181993210,8cyclictest29435-21ssh22:38:251
12181993210,8cyclictest1238-21ssh00:29:021
12181993210,8cyclictest1238-21ssh00:29:021
12181993210,7cyclictest30252-21diskmemload22:53:171
1225299315,18cyclictest35966irq/58-eth000:00:133
1225299315,18cyclictest35966irq/58-eth000:00:133
1225299314,17cyclictest35966irq/58-eth023:40:133
1222099313,18cyclictest27346irq/40-dwc2_hso22:00:122
1218199319,7cyclictest14616-21ssh23:40:351
1218199317,8cyclictest30252-21diskmemload21:53:441
12181993124,4cyclictest27568-21apt-get22:20:141
12181993122,4cyclictest29237-21apt-get20:35:121
12181993117,11cyclictest0-21swapper/123:15:111
12181993111,9cyclictest30252-21diskmemload00:36:231
12181993110,8cyclictest18573-21ssh23:45:051
12181993110,10cyclictest12289-21sh23:59:201
1216399311,24cyclictest11317-21ssh23:05:120
1222099303,11cyclictest27-21ksoftirqd/223:53:252
1222099301,23cyclictest30252-21diskmemload23:05:122
1222099301,18cyclictest16320-21ssh21:20:152
12181993023,3cyclictest1642-21apt-get20:40:131
1216399303,19cyclictest25937-21ssh00:25:150
1216399303,19cyclictest25937-21ssh00:25:150
1225299292,11cyclictest35966irq/58-eth022:35:133
12252992914,12cyclictest0-21swapper/322:10:133
1222099297,10cyclictest27-21ksoftirqd/222:15:112
1222099293,19cyclictest844-21munin-node20:55:142
1222099293,19cyclictest27346irq/40-dwc2_hso20:40:132
1222099293,16cyclictest27346irq/40-dwc2_hso19:50:112
1222099292,20cyclictest0-21swapper/219:20:142
1222099291,21cyclictest13100-21ssh22:30:102
1218199297,8cyclictest30252-21diskmemload22:15:161
1218199296,10cyclictest0-21swapper/119:45:241
12181992922,2cyclictest3992-21apt-get19:35:131
1225299283,18cyclictest0-21swapper/323:46:353
1222099283,8cyclictest131rcu_preempt00:20:142
1222099282,20cyclictest0-21swapper/219:25:142
1222099282,15cyclictest27346irq/40-dwc2_hso20:35:132
1222099281,15cyclictest9735-21ssh23:40:142
12181992814,11cyclictest11761-21kthreadcore19:40:221
12181992814,10cyclictest29359-21sed21:45:171
1225299271,19cyclictest35966irq/58-eth000:15:143
1225299271,18cyclictest35966irq/58-eth022:40:133
1222099276,15cyclictest27346irq/40-dwc2_hso22:20:132
1222099273,14cyclictest27346irq/40-dwc2_hso20:30:132
1222099273,14cyclictest27346irq/40-dwc2_hso20:30:132
1222099272,19cyclictest7766-21ssh22:45:132
1222099272,19cyclictest7766-21ssh22:45:132
1222099272,10cyclictest0-21swapper/219:55:142
1222099271,21cyclictest0-21swapper/223:15:352
1222099271,15cyclictest4187-21sh22:25:152
1218199278,8cyclictest31987-21usage_Anybus20:00:291
12181992718,6cyclictest20854-21apt-get21:00:131
12181992716,6cyclictest6547-21diskstats20:45:151
12181992714,10cyclictest17823-21kernelversion19:15:191
1225299263,17cyclictest0-21swapper/323:33:353
12252992615,7cyclictest0-21swapper/323:20:153
12252992614,7cyclictest35966irq/58-eth023:15:113
1222099261,14cyclictest20355-21ssh23:10:142
1222099261,14cyclictest20355-21ssh23:10:142
1222099260,11cyclictest10943-21ssh23:20:352
1222099260,10cyclictest9209-21ssh23:55:362
12181992622,2cyclictest21990-21apt-get19:20:131
12181992619,3cyclictest24455-21apt-get20:30:121
12181992619,3cyclictest24455-21apt-get20:30:121
12181992614,9cyclictest8776-21switchtime20:10:271
12181992614,9cyclictest15846-21ntp_states19:10:271
12181992613,10cyclictest13547-21switchtime20:15:271
1216399261,20cyclictest24090-21ssh23:30:130
1216399261,19cyclictest4176-21ssh22:25:140
12252992516,6cyclictest35966irq/58-eth023:10:123
12252992516,6cyclictest35966irq/58-eth023:10:123
1222099252,14cyclictest22159-21ssh22:35:132
12220992513,9cyclictest16432-21ssh00:35:352
1225299243,13cyclictest0-21swapper/323:37:353
1225299243,13cyclictest0-21swapper/323:37:353
12252992418,3cyclictest0-21swapper/300:25:133
12252992418,3cyclictest0-21swapper/300:25:133
1222099241,8cyclictest20713-21ssh00:03:352
1222099241,8cyclictest20713-21ssh00:03:352
1222099241,5cyclictest0-21swapper/200:05:432
12220992412,5cyclictest3598-21ssh21:48:092
12220992411,8cyclictest25888-21/usr/sbin/munin00:25:322
12220992411,8cyclictest25888-21/usr/sbin/munin00:25:312
12181992413,8cyclictest14605-21grep20:50:211
12181992413,8cyclictest12228-21date19:10:021
12181992412,9cyclictest3834-21awk20:05:261
12181992412,9cyclictest30696-21usage_Anybus19:25:301
1216399247,8cyclictest0-21swapper/020:40:130
1216399246,10cyclictest0-21swapper/019:20:130
1216399242,16cyclictest21908-21apt-get00:05:030
12163992418,3cyclictest12-21ksoftirqd/020:05:130
1225299233,13cyclictest0-21swapper/322:58:103
1225299232,13cyclictest35966irq/58-eth021:55:143
1225299232,13cyclictest35966irq/58-eth021:55:143
1225299231,14cyclictest36066irq/59-eth021:50:133
1222099233,14cyclictest27346irq/40-dwc2_hso20:20:142
1222099232,17cyclictest27346irq/40-dwc2_hso00:15:142
1222099232,16cyclictest31425-21ssh00:10:162
1222099231,8cyclictest30252-21diskmemload23:39:352
1222099231,8cyclictest30252-21diskmemload23:39:352
1222099231,18cyclictest0-21swapper/222:05:152
1222099231,18cyclictest0-21swapper/222:05:152
1222099231,16cyclictest0-21swapper/219:35:132
12220992311,10cyclictest12076-21tr21:35:202
1216399239,3cyclictest30252-21diskmemload22:58:210
1216399235,8cyclictest0-21swapper/019:35:130
1216399231,17cyclictest13303-21ssh00:00:140
1216399231,17cyclictest13303-21ssh00:00:140
1225299223,13cyclictest0-21swapper/322:53:353
12252992216,4cyclictest35966irq/58-eth022:05:143
12252992216,4cyclictest35966irq/58-eth022:05:143
1222099229,11cyclictest2426-1kworker/u9:022:50:302
1222099222,5cyclictest7370-21/usr/sbin/munin21:15:162
1222099221,5cyclictest7540-21kthreadcore21:50:222
1222099221,16cyclictest27346irq/40-dwc2_hso19:30:112
1222099221,16cyclictest0-21swapper/222:57:352
1216399222,16cyclictest13413-21ssh23:40:260
1216399222,14cyclictest29015-21ssh21:25:280
12163992215,5cyclictest0-21swapper/022:50:240
12163992215,4cyclictest22121-21sh22:15:250
1216399221,17cyclictest20331-21find23:10:120
1216399221,17cyclictest20331-21find23:10:120
1216399221,10cyclictest786-21df23:35:150
1216399221,10cyclictest786-21df23:35:150
12163992210,9cyclictest1391-21cron20:40:010
1225299212,10cyclictest0-21swapper/322:15:123
12252992115,2cyclictest0-21swapper/300:30:133
1222099219,5cyclictest4754-21kthreadcore00:30:232
12220992113,6cyclictest0-21swapper/219:10:282
1222099211,17cyclictest23230-21tr21:40:232
1222099211,16cyclictest24360-21ssh23:30:162
1222099211,10cyclictest13356-21kworker/2:019:45:192
1222099210,5cyclictest14550-21gpgv20:20:012
1216399219,8cyclictest28076-21df_inode20:00:130
1216399214,5cyclictest131rcu_preempt21:50:130
1216399213,13cyclictest0-21swapper/020:20:130
1216399211,16cyclictest9960-21find20:15:110
1216399211,16cyclictest2501-21ssh22:40:270
12163992111,5cyclictest12-21ksoftirqd/000:15:140
12163992110,8cyclictest15675-21uptime00:35:290
12163992110,8cyclictest11929-21uptime22:45:300
12163992110,8cyclictest11929-21uptime22:45:300
12163992110,5cyclictest12-21ksoftirqd/019:55:130
1225299203,11cyclictest0-21swapper/323:06:353
12252992015,2cyclictest0-21swapper/319:15:123
12252992015,2cyclictest0-21swapper/300:05:143
12252992014,2cyclictest0-21swapper/321:40:133
12252992010,8cyclictest0-21swapper/322:04:423
12252992010,8cyclictest0-21swapper/321:46:583
12252992010,7cyclictest35966irq/58-eth000:10:153
1222099203,8cyclictest25302-21acpi21:25:132
1222099202,8cyclictest21149-21kthreadcore20:25:202
1222099202,15cyclictest30809-21kthreadcore20:00:202
1222099202,13cyclictest27346irq/40-dwc2_hso22:10:132
1222099201,9cyclictest4632-21kthreadcore21:30:212
1222099201,9cyclictest29493-21unixbench_multi21:05:302
1222099201,9cyclictest24125-21latency21:00:222
1222099201,5cyclictest20783-21switchtime19:15:282
1222099201,10cyclictest31955-21ssh22:40:192
1222099201,10cyclictest10673-21kthreadcore19:40:212
1216399206,5cyclictest30252-21diskmemload23:50:190
1216399203,11cyclictest0-21swapper/022:20:120
12163992011,6cyclictest0-21swapper/021:35:300
12163992010,8cyclictest479-21sed23:15:230
1216399200,16cyclictest14525-21find21:55:120
1216399200,16cyclictest14525-21find21:55:120
1225299196,11cyclictest0-21swapper/323:25:283
1225299192,10cyclictest0-21swapper/320:00:113
12252991912,3cyclictest0-21swapper/319:30:113
1225299191,12cyclictest0-21swapper/320:15:283
1225299191,11cyclictest0-21swapper/319:45:013
1225299191,11cyclictest0-21swapper/319:35:153
12252991910,4cyclictest0-21swapper/321:05:133
1222099191,9cyclictest6840-21irqcore20:45:182
1222099191,15cyclictest30796-21fschecks_count21:10:152
1222099191,10cyclictest14876-21switchtime20:50:262
1216399193,3cyclictest131rcu_preempt21:05:130
1216399192,13cyclictest18704-21ssh23:45:140
1216399191,14cyclictest5971-21ssh23:00:240
1216399191,13cyclictest30252-21diskmemload22:30:250
1225299189,7cyclictest0-21swapper/321:20:223
1225299189,6cyclictest0-21swapper/322:20:463
1225299182,10cyclictest0-21swapper/319:10:053
12252991812,4cyclictest0-21swapper/300:20:133
12252991812,3cyclictest0-21swapper/320:30:123
12252991812,3cyclictest0-21swapper/320:30:123
12252991812,2cyclictest0-21swapper/320:55:123
1222099181,9cyclictest5378-21hwlatdetect20:10:152
1222099181,15cyclictest12227-21apt-key19:10:022
1222099181,14cyclictest1463-21kthreadcore20:05:202
1216399189,7cyclictest2499-21memory19:30:200
1216399189,6cyclictest18581-21perl19:50:150
1216399185,4cyclictest0-21swapper/021:00:130
1216399183,9cyclictest0-21swapper/020:25:120
1216399183,12cyclictest6771-21ps20:45:170
1216399181,9cyclictest0-21swapper/023:20:150
1216399181,14cyclictest7881-21irqrtprio21:15:200
1216399181,13cyclictest6754-21users21:30:310
1216399181,13cyclictest4411-21grep22:05:310
1216399181,13cyclictest4411-21grep22:05:310
1216399181,13cyclictest32621-21ntp_192.168.11421:45:250
1225299179,6cyclictest0-21swapper/322:26:223
1225299179,6cyclictest0-21swapper/321:35:193
1225299179,5cyclictest0-21swapper/322:45:513
1225299179,5cyclictest0-21swapper/322:45:513
1225299176,8cyclictest0-21swapper/320:25:123
1225299173,10cyclictest35966irq/58-eth020:05:133
1225299172,9cyclictest0-21swapper/320:45:143
1225299172,9cyclictest0-21swapper/319:45:133
12252991713,2cyclictest0-21swapper/319:25:133
1225299171,13cyclictest35966irq/58-eth021:30:113
12252991711,3cyclictest0-21swapper/320:50:103
12252991710,2cyclictest0-21swapper/319:50:113
1216399179,6cyclictest1-21systemd00:33:320
1216399179,5cyclictest27044-21wc19:25:170
1216399178,7cyclictest28249-21unixbench_multi20:30:280
1216399178,7cyclictest28249-21unixbench_multi20:30:280
1216399175,3cyclictest131rcu_preempt23:55:190
1216399171,9cyclictest13749-21kthreadcore19:10:200
1216399171,13cyclictest11259-21apt-get20:50:110
1225299167,6cyclictest0-21swapper/320:15:013
1225299165,8cyclictest0-21swapper/321:00:123
12252991612,2cyclictest0-21swapper/319:55:123
1225299161,12cyclictest0-21swapper/320:20:313
1216399167,6cyclictest10714-21cut19:40:210
1216399166,2cyclictest131rcu_preempt19:45:150
1216399162,11cyclictest9836-21latency_hist20:15:010
1216399161,12cyclictest2029-21ps21:10:290
1216399161,12cyclictest17887-21kthreadcore19:15:190
1225299158,4cyclictest0-21swapper/320:40:133
12252991511,2cyclictest0-21swapper/319:20:133
1216399158,4cyclictest12054-21apt-key19:10:020
1225299144,7cyclictest0-21swapper/320:35:223
1225299101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional