You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 02:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 25, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1573399542,40cyclictest28860-21ssh21:25:121
1574499422,27cyclictest9892-21ssh23:20:142
1572699423,35cyclictest13168-21ssh22:10:120
1572699411,36cyclictest31550-21ssh23:50:130
1572699403,34cyclictest29962-21ssh00:25:130
1572699403,34cyclictest29962-21ssh00:25:130
1574499398,25cyclictest15554-21ssh00:35:132
1573399389,18cyclictest0-21swapper/100:35:121
1573399368,5cyclictest21-21ksoftirqd/123:40:141
1572699363,30cyclictest27856-21sh23:30:120
1572699354,26cyclictest6279-21ssh23:00:130
15751993419,9cyclictest35966irq/58-eth023:10:113
1574499341,17cyclictest20840-21ssh23:43:242
1575199333,16cyclictest35966irq/58-eth022:20:113
1572699333,24cyclictest12-21ksoftirqd/019:55:120
1573399323,17cyclictest31239-21ssh22:20:121
1572699323,22cyclictest29608-21ssh22:55:130
1575199317,12cyclictest35966irq/58-eth022:15:133
1574499314,22cyclictest10356-21kworker/u8:021:43:232
1573399313,22cyclictest9872-21ssh23:20:131
1572699314,23cyclictest25958-21ssh22:35:150
1572699314,23cyclictest25958-21ssh22:35:140
1572699311,21cyclictest20597-21ssh22:50:110
1574499305,16cyclictest27346irq/40-dwc2_hso21:45:152
1574499303,17cyclictest27346irq/40-dwc2_hso21:25:132
1572699304,20cyclictest12-21ksoftirqd/019:40:140
1572699303,22cyclictest23580-21ssh21:40:140
1572699302,23cyclictest22241-21ssh22:15:140
1572699302,23cyclictest12-21ksoftirqd/021:50:130
15726993012,12cyclictest131rcu_preempt20:10:120
15751992914,12cyclictest35966irq/58-eth022:30:153
15751992914,12cyclictest35966irq/58-eth022:30:143
1574499293,21cyclictest0-21swapper/223:54:242
1574499293,17cyclictest27346irq/40-dwc2_hso19:30:122
1573399293,19cyclictest3973-21ssh22:05:141
1573399292,19cyclictest18845-21ssh23:05:241
1574499283,10cyclictest0-21swapper/222:26:242
1574499282,18cyclictest27346irq/40-dwc2_hso22:05:142
1572699284,20cyclictest21682-21sh23:07:250
1572699284,18cyclictest898-21ssh23:15:140
1572699284,15cyclictest20981-21ssh00:20:130
1572699282,21cyclictest18133-21sed21:35:240
1574499272,21cyclictest21519-21kthreadcore19:15:192
1574499271,6cyclictest1584466ssh23:05:172
1573399272,19cyclictest27381-21ssh00:22:241
1573399272,19cyclictest18914-21ssh23:25:141
1573399272,18cyclictest1246-21diskmemload21:43:231
1572699272,20cyclictest2610-21ssh22:40:140
1575199262,14cyclictest35966irq/58-eth023:05:123
15751992614,8cyclictest35966irq/58-eth023:30:123
1575199261,16cyclictest35966irq/58-eth000:00:123
1575199261,15cyclictest0-21swapper/323:00:123
1574499263,20cyclictest21132-21sendmail_mailqu00:00:262
1574499263,18cyclictest0-21swapper/200:23:452
1574499262,19cyclictest3042-21ssh22:04:242
1573399262,19cyclictest15357-21ssh21:51:561
1572699263,18cyclictest17384-21ssh00:00:120
1572699262,20cyclictest9619-21ssh22:06:240
1575199254,15cyclictest35966irq/58-eth000:05:123
1575199253,11cyclictest35966irq/58-eth023:45:143
1574499252,21cyclictest4953-21unixbench_multi23:15:292
1574499252,20cyclictest28814-21kthreadcore20:30:202
1574499252,20cyclictest10238-21ntp_offset00:30:262
1574499252,16cyclictest0-21swapper/221:20:052
1574499251,9cyclictest23986-21ssh21:56:252
1574499251,15cyclictest8893-21timerwakeupswit20:40:282
1573399251,7cyclictest647-21sed22:55:211
15733992512,10cyclictest19467-21ssh21:55:191
1572699254,13cyclictest0-21swapper/000:15:140
1572699252,18cyclictest10677-21rm21:15:140
1572699252,17cyclictest8331-21diskmemload23:55:140
1572699251,21cyclictest16935-21sh22:30:150
1572699251,21cyclictest16935-21sh22:30:140
1575199246,14cyclictest35966irq/58-eth023:20:133
1575199242,18cyclictest35966irq/58-eth020:25:143
1575199242,17cyclictest35966irq/58-eth021:40:133
1575199242,17cyclictest0-21swapper/321:47:113
15751992415,6cyclictest0-21swapper/323:50:133
1574499243,15cyclictest27346irq/40-dwc2_hso23:10:102
1574499243,14cyclictest29820-21fschecks_time22:55:152
1574499243,14cyclictest11849-21sshd23:55:252
1574499242,7cyclictest0-21swapper/222:48:242
1574499242,13cyclictest5257-21tr00:10:222
1574499241,7cyclictest93066snmpd22:25:002
1574499241,20cyclictest14921-21kthreadcore20:15:202
1574499241,15cyclictest29217-21kthreadcore23:30:182
1574499241,15cyclictest20449-21needreboot19:45:232
1573399249,12cyclictest21-21ksoftirqd/120:20:121
1573399248,3cyclictest131rcu_preempt21:25:001
1573399241,20cyclictest14480-21ssh21:35:141
1572699246,12cyclictest26324-21ssh00:05:120
1572699243,10cyclictest131rcu_preempt23:40:010
15726992415,3cyclictest0-21swapper/020:50:110
1575199234,12cyclictest35966irq/58-eth022:40:123
1575199233,17cyclictest35966irq/58-eth022:35:133
1575199233,17cyclictest35966irq/58-eth022:35:123
1575199231,10cyclictest0-21swapper/300:15:143
15751992310,10cyclictest0-21swapper/321:55:023
1574499236,9cyclictest18214-21wc20:50:262
1574499232,6cyclictest0-21swapper/221:51:552
1574499232,16cyclictest0-21swapper/223:49:252
1574499232,16cyclictest0-21swapper/222:53:242
1574499232,16cyclictest0-21swapper/200:28:242
1574499232,16cyclictest0-21swapper/200:28:232
1574499231,19cyclictest7016-21kthreadcore23:00:192
1574499231,19cyclictest23594-21irqcore20:25:182
1574499231,19cyclictest17079-21fschecks_count22:30:162
1574499231,19cyclictest17079-21fschecks_count22:30:162
1574499231,15cyclictest30785-21kthreadcore21:05:182
1574499231,13cyclictest10910-21kthreadcore20:45:212
1573399239,3cyclictest131rcu_preempt22:15:181
15733992316,4cyclictest211ksoftirqd/120:05:121
1572699234,8cyclictest0-21swapper/022:00:020
1572699232,18cyclictest1909-21ls21:10:150
1572699232,15cyclictest6568-21ssh00:30:140
1572699232,15cyclictest17192-21ntp_192.168.11423:40:240
1572699231,17cyclictest5070-21latency_hist21:30:020
1575199229,9cyclictest35966irq/58-eth000:20:143
1575199222,17cyclictest35966irq/58-eth023:15:133
1575199222,13cyclictest0-21swapper/300:25:143
1575199222,13cyclictest0-21swapper/300:25:143
1574499223,16cyclictest30374-21cpuspeed19:25:152
1574499223,16cyclictest30374-21cpuspeed19:25:142
1574499223,12cyclictest20159-21uname20:55:192
1574499223,12cyclictest20159-21uname20:55:192
1574499222,17cyclictest19844-21kthreadcore20:20:192
1574499222,15cyclictest14481-21ssh21:35:142
15744992211,9cyclictest2154-21ssh22:40:002
15744992211,9cyclictest2154-21ssh22:40:002
1574499221,10cyclictest930-21snmpd23:25:122
1573399229,10cyclictest29230-21ssh22:35:221
1573399229,10cyclictest29230-21ssh22:35:221
1573399224,9cyclictest15712-21cyclictest23:45:141
1573399224,3cyclictest131rcu_preempt22:10:041
1573399223,15cyclictest31533-21apt-get23:50:111
1573399222,17cyclictest8381-21cat20:40:211
1573399221,17cyclictest7896-21sh22:25:131
1573399220,18cyclictest24142-21ls22:50:231
1572699222,14cyclictest3761-21memory21:45:250
1572699221,16cyclictest29300-21cut21:05:150
1575199219,9cyclictest0-21swapper/321:31:273
1574499214,9cyclictest9465-21kthreadcore19:35:192
1574499213,11cyclictest11405-21sed21:15:192
1574499212,16cyclictest2229-21kthreadcore20:00:222
1574499212,15cyclictest10428-21latency_hist21:15:022
1574499212,11cyclictest9132-21irqcore20:10:172
1574499212,11cyclictest29066-21grep21:05:012
1574499212,11cyclictest22800-21ps22:15:192
1574499211,6cyclictest0-21swapper/220:05:122
1574499211,17cyclictest27291-21irqrtprio19:55:172
1574499211,17cyclictest14324-21kthreadcore00:15:222
15744992110,9cyclictest11133-21ssh21:31:302
1573399212,16cyclictest27917-21apt-get20:30:121
1573399212,16cyclictest19356-21grep21:20:011
1573399211,8cyclictest2495-21apt-get00:10:021
1572699218,3cyclictest131rcu_preempt21:20:040
1572699215,5cyclictest131rcu_preempt19:40:010
1572699212,14cyclictest9886-21ssh23:20:130
1572699212,11cyclictest0-21swapper/021:30:140
1572699212,10cyclictest14992-21kthreadcore22:45:250
15726992111,7cyclictest14474-21cron20:50:020
1575199209,8cyclictest0-21swapper/322:54:223
15751992010,8cyclictest0-21swapper/321:18:283
15751992010,4cyclictest0-21swapper/300:35:133
1574499203,9cyclictest29347-21grep00:05:222
1574499202,11cyclictest349-21diskstats20:35:142
1574499202,11cyclictest3088-21sed22:40:192
1574499202,10cyclictest26220-21kthreadcore19:20:202
1574499201,16cyclictest19357-21ntp_kernel_pll_19:10:242
1574499201,11cyclictest13159-21ssh22:10:122
1574499201,10cyclictest4578-21cpu23:35:152
1573399203,14cyclictest22975-21kthreadcore19:50:191
1573399202,16cyclictest21936-21kthreadcore20:55:211
1573399202,16cyclictest21936-21kthreadcore20:55:211
1573399202,15cyclictest941-21cpu23:15:141
1573399202,15cyclictest9021-21kthreadcore19:35:191
1573399202,15cyclictest29652-21kthreadcore19:55:221
1573399202,15cyclictest1800-21fschecks_count21:10:151
1573399202,15cyclictest15229-21kthreadcore20:15:211
1573399201,5cyclictest8247-21ntp_states23:35:261
15733992012,5cyclictest26761-21ssh00:05:171
15733992011,6cyclictest15630-21ssh00:15:281
1573399201,10cyclictest402-21tr20:00:191
15733992010,8cyclictest5449-21apt-get21:30:131
1572699209,9cyclictest26944-21cat23:45:310
1572699202,15cyclictest7880-21apt-get22:25:130
15726992011,7cyclictest12-21ksoftirqd/020:15:140
1572699201,15cyclictest24976-21kthreadcore23:10:160
1572699200,17cyclictest2396-21ls22:20:220
1575199193,11cyclictest0-21swapper/321:37:253
1575199192,12cyclictest0-21swapper/322:05:243
15751991912,4cyclictest0-21swapper/323:35:183
15751991912,4cyclictest0-21swapper/322:00:213
1575199191,13cyclictest0-21swapper/320:45:313
1575199191,13cyclictest0-21swapper/319:25:163
1575199191,13cyclictest0-21swapper/319:25:153
15751991910,7cyclictest35966irq/58-eth022:10:113
15751991910,6cyclictest35966irq/58-eth023:55:143
15751991910,6cyclictest0-21swapper/323:40:143
1574499191,16cyclictest12882-21irqrtprio19:40:182
1574499191,15cyclictest22100-21fschecks_count19:50:152
1573399199,7cyclictest18400-21kthreadcore22:30:211
1573399199,7cyclictest18400-21kthreadcore22:30:201
1573399193,13cyclictest25563-21cpu19:20:131
1573399192,5cyclictest8550-21kthreadcore00:30:231
1573399192,15cyclictest2519-21kthreadcore20:35:201
1573399192,15cyclictest18290-21kthreadcore19:10:201
1573399192,13cyclictest24228-21ssh23:10:121
1573399191,15cyclictest20741-21cpu19:15:141
1573399191,14cyclictest7851-21apt-get23:55:031
15733991910,6cyclictest11622-21ssh22:45:141
1572699195,11cyclictest658-21irqcore20:35:170
1572699193,13cyclictest32496-21idleruntime-cro20:35:000
1572699192,3cyclictest131rcu_preempt19:20:010
1572699192,15cyclictest17455-21irqcore19:45:180
1572699191,8cyclictest17450-21kthreadcore00:35:200
1572699191,16cyclictest16087-21fschecks_time19:10:150
1572699191,15cyclictest0-21swapper/020:25:140
1575199189,6cyclictest0-21swapper/322:56:533
1575199182,11cyclictest0-21swapper/322:27:153
1575199182,10cyclictest0-21swapper/320:35:143
1575199182,10cyclictest0-21swapper/319:40:143
1575199182,10cyclictest0-21swapper/300:10:043
15751991813,2cyclictest0-21swapper/321:10:113
1575199181,13cyclictest0-21swapper/320:40:153
1575199181,11cyclictest0-21swapper/300:30:123
15751991810,6cyclictest0-21swapper/322:45:203
15751991810,3cyclictest0-21swapper/321:57:053
1573399185,10cyclictest30361-21ssh00:25:191
1573399185,10cyclictest30361-21ssh00:25:181
1573399182,14cyclictest6658-21irqcore23:00:181
1573399182,14cyclictest17354-21ssh00:00:111
1573399182,13cyclictest29687-21kthreadcore22:00:221
1573399182,13cyclictest28087-21fschecks_time23:30:161
1573399182,13cyclictest17545-21irqcore19:45:181
15733991810,6cyclictest7559-21sshd21:48:241
1572699185,10cyclictest6926-21sh00:10:300
1572699185,10cyclictest4514-21irqrtprio20:05:180
1572699182,4cyclictest131rcu_preempt22:00:140
1572699182,14cyclictest22996-21ps20:55:270
1572699182,14cyclictest22996-21ps20:55:260
1572699182,13cyclictest27154-21tr19:20:200
1572699182,13cyclictest24476-21fschecks_time21:00:160
1572699181,12cyclictest20347-21kthreadcore23:25:210
1572699181,11cyclictest5338-21ls20:40:150
1575199179,6cyclictest0-21swapper/321:21:423
1575199172,9cyclictest0-21swapper/320:00:123
1575199172,8cyclictest35966irq/58-eth019:10:113
15751991711,4cyclictest0-21swapper/323:25:043
15751991711,4cyclictest0-21swapper/321:25:183
1575199171,13cyclictest0-21swapper/321:05:163
1575199171,12cyclictest35966irq/58-eth019:20:133
1574499171,8cyclictest15838-21latency_hist19:10:022
1573399178,7cyclictest13418-21cut19:40:191
1573399173,12cyclictest6516-21grep19:30:281
1573399172,13cyclictest9034-21irqcore20:10:171
1573399172,12cyclictest15043-21grep20:50:171
1573399172,10cyclictest0-21swapper/121:05:121
15733991710,5cyclictest6275-21ssh22:40:261
1572699173,11cyclictest22090-21tune2fs19:50:160
1572699171,13cyclictest31801-21sed20:00:160
1572699171,12cyclictest4607-21kthreadcore19:30:190
1572699171,12cyclictest30501-21grep19:25:160
1572699171,12cyclictest30501-21grep19:25:160
1572699171,11cyclictest19577-21kthreadcore20:20:180
1575199162,9cyclictest0-21swapper/320:15:153
1575199162,10cyclictest0-21swapper/320:10:133
1575199161,12cyclictest0-21swapper/319:55:153
1573399164,10cyclictest11543-21kthreadcore20:45:211
1573399162,12cyclictest26189-21kthreadcore21:00:201
1573399162,11cyclictest15655-21latency_hist19:10:031
1573399161,11cyclictest1265-21missed_timers19:25:221
1573399161,11cyclictest1265-21missed_timers19:25:221
15733991610,4cyclictest0-21swapper/120:25:161
1575199152,9cyclictest0-21swapper/320:20:123
1575199151,9cyclictest0-21swapper/319:45:203
1575199143,8cyclictest0-21swapper/321:00:153
1575199142,6cyclictest0-21swapper/319:35:133
1575199141,11cyclictest0-21swapper/319:15:213
1575199141,10cyclictest0-21swapper/319:50:203
1572699140,10cyclictest15760-21latency_hist19:10:020
1575199132,7cyclictest0-21swapper/321:00:023
1575199132,7cyclictest0-21swapper/321:00:013
1575199132,6cyclictest0-21swapper/320:50:133
1575199131,8cyclictest35966irq/58-eth020:30:143
1575199131,7cyclictest0-21swapper/320:05:213
1575199131,10cyclictest0-21swapper/319:30:163
1575199111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional