You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 13:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Sep 05, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3013599682,35cyclictest19370-21ssh00:10:150
30168995729,19cyclictest35966irq/58-eth021:40:113
3013599504,6cyclictest131rcu_preempt00:15:120
3015599491,26cyclictest6344-21ssh23:45:122
3015599480,24cyclictest0-21swapper/220:40:122
3014699473,33cyclictest73-21kswapd000:20:151
30155994521,17cyclictest0-21swapper/219:10:102
3014699454,30cyclictest14016-21ssh00:25:131
3014699432,33cyclictest29424-21ssh21:35:121
30168994122,13cyclictest35966irq/58-eth000:25:143
30155994135,3cyclictest27-21ksoftirqd/200:15:132
3014699411,28cyclictest15876-21diskmemload23:30:111
3013599414,34cyclictest14159-21ssh22:20:140
30146994010,20cyclictest21-21ksoftirqd/123:10:131
3016899382,23cyclictest35966irq/58-eth021:50:123
3015599382,9cyclictest536-21usb-storage22:15:102
30155993810,17cyclictest0-21swapper/222:40:112
3015599379,16cyclictest630-21Async09:36:312
3014699376,21cyclictest201rcuc/121:25:131
3015599366,14cyclictest3728-21ssh22:50:102
3015599364,25cyclictest2081-21ssh21:20:132
3013599368,7cyclictest131rcu_preempt21:00:110
3016899353,19cyclictest35966irq/58-eth021:15:123
3015599353,9cyclictest0-21swapper/223:17:092
3013599354,28cyclictest10317-21ssh00:05:120
3013599351,31cyclictest6354-21ssh23:45:120
3014699342,20cyclictest31028-21ssh23:05:121
3013599348,10cyclictest131rcu_preempt19:45:120
3013599343,27cyclictest22984-21ssh00:30:120
3013599343,27cyclictest16279-21ssh21:10:130
30155993321,8cyclictest17028-21apt-get19:30:112
3013599334,26cyclictest27085-21ssh22:45:120
3013599331,28cyclictest2060-21ssh21:20:120
3015599323,19cyclictest27-21ksoftirqd/200:05:122
3015599321,9cyclictest12360-21cut19:25:152
3015599318,9cyclictest0-21swapper/221:25:132
3015599317,12cyclictest0-21swapper/223:37:032
3015599315,14cyclictest32311-21ssh22:30:152
3015599315,14cyclictest0-21swapper/223:20:122
30155993112,13cyclictest19346-21apt-get00:10:142
30135993122,6cyclictest12-21ksoftirqd/022:10:120
3016899301,15cyclictest35966irq/58-eth022:25:123
3015599306,13cyclictest0-21swapper/223:14:152
3015599305,21cyclictest0-21swapper/220:05:122
3015599305,17cyclictest0-21swapper/221:10:132
3015599305,14cyclictest0-21swapper/221:43:222
3015599305,13cyclictest27346irq/40-dwc2_hso19:20:122
3015599302,17cyclictest8867-21ssh22:35:122
30135993025,2cyclictest20577-21ssh23:35:110
3016899292,15cyclictest35966irq/58-eth021:30:123
30168992915,10cyclictest35966irq/58-eth022:00:143
3015599296,13cyclictest0-21swapper/223:06:512
3015599296,13cyclictest0-21swapper/223:01:162
3015599296,12cyclictest0-21swapper/221:59:442
3015599296,12cyclictest0-21swapper/221:39:502
3015599296,12cyclictest0-21swapper/200:22:322
3015599293,5cyclictest27-21ksoftirqd/200:30:232
3015599293,20cyclictest0-21swapper/222:10:132
3015599292,21cyclictest0-21swapper/223:44:342
3015599292,20cyclictest0-21swapper/221:00:122
3015599292,20cyclictest0-21swapper/220:55:132
3015599292,15cyclictest27-21ksoftirqd/223:56:102
3015599291,17cyclictest27346irq/40-dwc2_hso23:25:142
30146992918,7cyclictest15876-21diskmemload23:15:131
3013599293,21cyclictest12-21ksoftirqd/020:50:130
30135992916,9cyclictest131rcu_preempt23:55:120
3015599286,13cyclictest0-21swapper/222:56:562
30155992816,8cyclictest27-21ksoftirqd/200:03:262
3014699289,4cyclictest131rcu_preempt22:55:011
3014699282,20cyclictest3050-21ssh23:41:331
3014699282,20cyclictest24205-21ssh21:50:131
3013599283,19cyclictest894-21ssh21:55:130
3016899273,16cyclictest0-21swapper/322:54:343
3015599276,11cyclictest27346irq/40-dwc2_hso20:20:132
3015599275,6cyclictest27-21ksoftirqd/222:20:202
3015599274,7cyclictest16748-21ssh22:02:462
3015599273,4cyclictest27-21ksoftirqd/221:50:222
3015599272,10cyclictest27-21ksoftirqd/219:40:222
30155992719,5cyclictest14705-21ssh23:30:202
3015599271,15cyclictest19108-21ssh22:05:112
3013599272,19cyclictest9927-21ssh22:52:040
30135992710,13cyclictest12-21ksoftirqd/023:50:110
3015599264,4cyclictest27-21ksoftirqd/221:15:232
3015599264,12cyclictest0-21swapper/221:48:382
3015599263,14cyclictest0-21swapper/219:50:242
3015599262,15cyclictest27346irq/40-dwc2_hso00:35:132
3015599261,18cyclictest15430-21ssh23:50:112
3014699264,18cyclictest201rcuc/122:25:121
3013599263,19cyclictest4973-21ssh22:15:130
3013599263,19cyclictest12931-21ssh22:55:130
3013599262,20cyclictest1165-21ssh00:00:130
30135992614,10cyclictest12-21ksoftirqd/019:15:150
3013599261,15cyclictest0-21swapper/019:30:120
3016899252,15cyclictest0-21swapper/323:37:343
30168992513,7cyclictest35966irq/58-eth023:05:133
3016899251,15cyclictest0-21swapper/321:25:133
3015599253,14cyclictest27346irq/40-dwc2_hso20:00:122
3015599252,14cyclictest27-21ksoftirqd/222:26:442
3014699252,18cyclictest23744-21diskmemload23:54:331
30146992514,7cyclictest21-21ksoftirqd/119:25:121
3013599252,20cyclictest12-21ksoftirqd/023:10:130
3015599244,11cyclictest2737-21entropy19:15:172
3015599244,10cyclictest27-21ksoftirqd/221:30:302
3013599242,17cyclictest25857-21rm23:20:140
3013599241,18cyclictest23176-21apt-get20:10:140
3016899236,11cyclictest0-21swapper/323:42:143
3016899233,13cyclictest0-21swapper/323:54:343
3016899233,13cyclictest0-21swapper/323:17:093
3015599232,4cyclictest271ksoftirqd/221:05:342
3015599232,4cyclictest271ksoftirqd/221:05:342
3015599231,9cyclictest9031-21grep20:25:292
30155992312,8cyclictest19436-21cron20:40:012
30155992310,9cyclictest29023-21cron20:50:022
3014699239,3cyclictest131rcu_preempt20:15:021
3014699238,10cyclictest30745-21diskmemload23:20:341
3014699237,5cyclictest131rcu_preempt22:15:151
3014699235,9cyclictest19146-21kthreadcore21:10:211
3014699234,10cyclictest20272-21ssh21:30:121
3013599233,14cyclictest12-21ksoftirqd/020:05:120
3013599232,6cyclictest0-21swapper/022:41:340
3013599232,16cyclictest32096-21ssh00:35:130
3016899223,13cyclictest0-21swapper/322:38:343
3016899221,13cyclictest0-21swapper/321:45:113
3016899220,16cyclictest35966irq/58-eth021:05:123
3016899220,16cyclictest35966irq/58-eth021:05:113
3015599228,5cyclictest17017-21kthreadcore00:25:222
3015599224,14cyclictest0-21swapper/220:10:142
3015599223,4cyclictest27-21ksoftirqd/219:45:332
3015599222,15cyclictest27346irq/40-dwc2_hso20:30:112
30155992212,8cyclictest0-21swapper/220:50:282
3015599221,11cyclictest31291-21grep20:15:212
3014699222,14cyclictest2499-21ssh23:25:151
3014699221,9cyclictest21612-21sendmail23:00:011
3014699221,16cyclictest25382-21ssh21:15:131
3014699221,15cyclictest10007-21ssh22:00:141
3013599225,13cyclictest22020-21rm23:00:130
3013599224,16cyclictest9988-21ssh22:00:130
3013599222,17cyclictest2457-21find23:25:130
3013599221,18cyclictest18190-21ls20:35:230
3013599221,16cyclictest24416-21cut21:50:150
3016899212,11cyclictest0-21swapper/300:10:153
30168992110,9cyclictest0-21swapper/300:30:183
3015599216,12cyclictest10641-21kthreadcore19:55:202
3014699215,5cyclictest131rcu_preempt23:35:201
3014699211,9cyclictest3381-21basename22:50:021
30146992115,4cyclictest18407-21kworker/1:200:10:151
3014699211,18cyclictest3636-21ls20:20:201
30146992111,7cyclictest27830-21cron22:10:011
3014699211,16cyclictest14179-21ssh22:20:151
3014699211,10cyclictest32251-21ssh00:35:151
3014699210,10cyclictest32379-21cpuspeed22:30:171
3013599217,7cyclictest12-21ksoftirqd/021:46:240
3013599212,15cyclictest11223-21ssh21:25:140
3013599211,3cyclictest131rcu_preempt21:39:050
30135992112,7cyclictest0-21swapper/021:10:010
30135992112,7cyclictest0-21swapper/021:10:010
3016899209,7cyclictest35966irq/58-eth023:00:123
3016899209,6cyclictest0-21swapper/321:37:423
3016899203,11cyclictest0-21swapper/323:24:343
3016899202,12cyclictest0-21swapper/323:49:353
3016899202,11cyclictest35966irq/58-eth022:15:113
3016899202,10cyclictest0-21swapper/320:00:133
30168992010,8cyclictest0-21swapper/321:22:473
3015599201,7cyclictest0-21swapper/219:35:142
3014699209,8cyclictest6493-21if_eth021:40:161
30146992013,4cyclictest25607-21ntp_192.168.11423:00:251
3014699201,15cyclictest24408-21grep20:45:011
30146992010,8cyclictest13671-21df_inode20:00:141
3014699200,10cyclictest11253-21tr22:35:201
3013599209,3cyclictest131rcu_preempt23:40:280
3013599206,4cyclictest27709-21usage_EthcTask22:25:300
3013599202,3cyclictest131rcu_preempt22:05:140
3013599202,12cyclictest16347-21ssh22:38:340
30135992014,4cyclictest0-21swapper/023:15:140
30135992011,6cyclictest13620-21cron00:25:010
3013599200,2cyclictest131rcu_preempt21:41:010
3016899199,7cyclictest0-21swapper/323:25:153
3016899198,6cyclictest0-21swapper/322:09:013
3016899193,11cyclictest0-21swapper/322:41:333
3016899193,11cyclictest0-21swapper/300:03:443
3016899192,10cyclictest0-21swapper/320:35:053
30168991913,4cyclictest0-21swapper/323:10:253
3016899191,13cyclictest0-21swapper/322:30:153
3016899191,13cyclictest0-21swapper/320:20:273
3016899191,12cyclictest0-21swapper/320:50:023
3016899191,12cyclictest0-21swapper/320:15:023
3015599196,4cyclictest27-21ksoftirqd/219:10:012
3015599196,4cyclictest27-21ksoftirqd/219:10:002
3014699197,9cyclictest5400-21diskmemload21:20:211
3014699194,3cyclictest131rcu_preempt22:40:251
3014699193,9cyclictest0-21swapper/119:20:131
3014699193,4cyclictest21-21ksoftirqd/121:55:211
3014699191,8cyclictest15062-21munin-plugin-st23:50:011
3014699191,8cyclictest10165-21ssh00:05:031
30146991911,6cyclictest31283-21sshd19:42:331
3014699191,16cyclictest10182-21threads21:00:261
3014699191,15cyclictest29390-21cut20:50:161
3014699191,15cyclictest27763-21kthreadcore23:55:211
3014699191,15cyclictest18576-21cut20:05:151
30146991911,5cyclictest1115-21/usr/sbin/munin00:00:111
3014699190,17cyclictest11547-21idleruntime21:05:161
3014699190,17cyclictest11547-21idleruntime21:05:161
3013599197,4cyclictest0-21swapper/020:45:240
3013599194,4cyclictest12-21ksoftirqd/021:15:300
3013599192,15cyclictest14069-21irqcore20:00:170
3013599191,15cyclictest15101-21ssh23:30:230
3013599191,10cyclictest32410-21ssh22:30:170
3016899189,6cyclictest0-21swapper/322:20:093
3016899183,10cyclictest0-21swapper/322:47:043
3016899183,10cyclictest0-21swapper/300:39:013
3016899182,11cyclictest0-21swapper/323:58:443
3016899182,11cyclictest0-21swapper/323:30:353
3016899182,10cyclictest0-21swapper/319:55:133
3016899181,12cyclictest0-21swapper/300:15:173
30168991810,6cyclictest0-21swapper/322:10:133
30168991810,6cyclictest0-21swapper/321:57:093
30168991810,6cyclictest0-21swapper/300:22:103
30168991810,6cyclictest0-21swapper/300:09:463
3014699188,7cyclictest20675-21cat19:30:261
3014699185,10cyclictest25821-21kthreadcore20:45:191
3014699181,7cyclictest10103-21cut20:30:141
30146991812,4cyclictest28791-21irqrtprio22:10:171
30146991810,6cyclictest8542-21sshd19:54:331
30146991810,6cyclictest3720-21sshd19:46:341
3013599182,9cyclictest151rcuc/020:15:120
3013599181,8cyclictest24532-21cat21:30:290
3013599181,8cyclictest14222-21fschecks_time00:25:160
30135991813,3cyclictest0-21swapper/019:20:250
30135991812,4cyclictest0-21swapper/020:40:240
3013599181,13cyclictest2195-21ntp_192.168.11423:05:240
3013599181,13cyclictest15390-21kthreadcore19:25:210
3013599181,12cyclictest31437-21cat19:45:010
3013599180,3cyclictest131rcu_preempt19:50:210
30168991711,4cyclictest0-21swapper/321:10:143
30168991711,3cyclictest0-21swapper/322:59:043
3014699179,6cyclictest0-21swapper/121:45:131
3014699176,8cyclictest32510-21grep00:15:301
3014699172,3cyclictest131rcu_preempt00:30:131
3014699172,10cyclictest0-21swapper/120:35:141
3014699171,10cyclictest9018-21idleruntime19:55:161
3013599179,6cyclictest1-21systemd20:21:440
3013599176,8cyclictest0-21swapper/020:30:120
3013599172,4cyclictest131rcu_preempt19:55:180
3013599171,7cyclictest22019-21sort19:35:150
3013599171,12cyclictest869-21cut19:10:190
30135991710,3cyclictest12-21ksoftirqd/020:55:130
3016899169,5cyclictest35966irq/58-eth020:40:133
3016899169,5cyclictest0-21swapper/319:54:413
3016899162,10cyclictest0-21swapper/320:15:123
3014699168,6cyclictest1903-21wc20:55:161
3014699168,5cyclictest31072-21cut19:10:161
3014699163,10cyclictest4686-21kthreadcore19:15:221
3014699162,11cyclictest21991-21fschecks_time19:35:151
3014699161,12cyclictest8484-21missed_timers20:25:231
3014699161,10cyclictest29896-21kthreadcore20:15:191
3013599161,8cyclictest5176-21find20:25:120
3016899151,11cyclictest0-21swapper/320:05:153
3016899146,5cyclictest35966irq/58-eth020:50:133
3016899142,6cyclictest0-21swapper/319:35:133
3016899141,8cyclictest35966irq/58-eth019:10:103
3016899141,11cyclictest0-21swapper/320:30:263
3016899141,10cyclictest0-21swapper/319:20:233
3014699141,10cyclictest30173-21sort19:10:011
3014699141,10cyclictest30173-21sort19:10:011
3016899131,9cyclictest0-21swapper/319:45:173
3016899131,7cyclictest0-21swapper/319:15:293
3016899131,5cyclictest0-21swapper/319:40:163
3016899131,10cyclictest0-21swapper/319:25:193
3013599133,7cyclictest30203-21latency_hist19:10:010
3013599133,7cyclictest30203-21latency_hist19:10:010
3016899123,7cyclictest0-21swapper/320:25:163
3016899122,7cyclictest0-21swapper/320:55:153
3016899121,6cyclictest0-21swapper/319:30:133
3016899111,7cyclictest0-21swapper/321:00:133
3016899101,6cyclictest0-21swapper/319:10:013
3016899101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional