You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 07:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 19, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
893129542,48sleep20-21swapper/219:06:482
893129542,48sleep20-21swapper/219:06:472
887729179,8sleep30-21swapper/319:06:043
887729179,8sleep30-21swapper/319:06:043
910729077,8sleep00-21swapper/019:09:030
910729077,8sleep00-21swapper/019:09:020
889028063,13sleep10-21swapper/119:06:151
889028063,13sleep10-21swapper/119:06:141
14852520,0sleep20-21swapper/223:05:092
44122470,0sleep30-21swapper/322:28:043
9332992827,1cyclictest0-21swapper/323:37:273
932399280,1cyclictest0-21swapper/123:09:271
9318992611,4cyclictest13-21ksoftirqd/021:48:400
932399244,4cyclictest3540-21latency_hist23:45:001
932399242,2cyclictest141rcu_preempt22:56:491
932399240,1cyclictest0-21swapper/122:21:501
9318992413,9cyclictest0-21swapper/000:36:280
932399233,5cyclictest31260-21tr19:55:131
931899236,7cyclictest21748-21idleruntime23:55:190
192212230,1sleep112445-21kworker/1:200:31:561
932399223,7cyclictest4836-21if_enp0s31f623:45:161
931899223,6cyclictest1860-21diskmemload23:12:420
932799211,7cyclictest32-21ksoftirqd/221:10:132
932399216,10cyclictest25-21ksoftirqd/122:14:571
9327992013,3cyclictest32-21ksoftirqd/223:34:142
32192200,0sleep30-21swapper/321:49:083
282412200,0chrt0-21swapper/123:59:421
9332991918,1cyclictest0-21swapper/323:31:183
932799194,2cyclictest141rcu_preempt00:02:032
932399199,3cyclictest10819-21tr20:20:121
932399193,8cyclictest21464-21apt-get19:35:141
9323991916,3cyclictest0-21swapper/122:15:361
931899196,3cyclictest200802sleep022:18:040
931899195,7cyclictest12660-21munin-run20:24:590
9318991910,2cyclictest30712-21munin-plugin-st19:54:590
9332991817,1cyclictest0-21swapper/322:02:523
9332991815,3cyclictest39-21ksoftirqd/321:58:243
932799181,1cyclictest0-21swapper/221:32:202
932399189,3cyclictest29399-21grep21:25:251
932399189,2cyclictest27584-21grep23:00:281
932399189,2cyclictest13603-21wc21:35:251
932399182,15cyclictest1860-21diskmemload00:26:491
9323991817,0cyclictest0-21swapper/120:10:131
931899181,4cyclictest0-21swapper/019:35:140
229872180,0sleep30-21swapper/323:55:363
933299170,1cyclictest0-21swapper/322:36:453
933299170,1cyclictest0-21swapper/322:36:443
9327991712,1cyclictest32-21ksoftirqd/222:01:302
932799170,1cyclictest0-21swapper/221:48:282
932799170,17cyclictest0-21swapper/221:20:152
932399176,6cyclictest23705-21cron22:40:011
932399176,6cyclictest23705-21cron22:40:001
9323991714,1cyclictest0-21swapper/123:34:061
9323991710,2cyclictest31050-21grep00:00:261
931899179,2cyclictest28367-21cat21:45:000
931899179,2cyclictest28319-21grep21:25:150
931899179,2cyclictest11643-21cut21:15:160
9318991710,2cyclictest16617-21grep00:30:220
933299162,13cyclictest0-21swapper/321:38:383
9332991612,3cyclictest12415-21ssh21:15:253
933299160,2cyclictest0-21swapper/321:20:323
932799169,4cyclictest7962-21cat22:10:242
9327991611,1cyclictest32-21ksoftirqd/222:49:532
932799160,1cyclictest141rcu_preempt21:39:242
932399161,13cyclictest23518-21wc00:15:181
931899169,2cyclictest25559-21grep19:40:260
931899168,3cyclictest23682-21wc20:45:210
931899168,3cyclictest17169-21ssh21:37:570
9318991611,2cyclictest13-21ksoftirqd/000:24:070
933299153,9cyclictest2743-21ssh23:25:123
933299151,2cyclictest16581-21kworker/u8:223:46:383
933299150,2cyclictest0-21swapper/300:09:553
933299150,12cyclictest0-21swapper/322:15:173
932799156,5cyclictest225-21systemd-journal23:25:012
932799152,1cyclictest141rcu_preempt22:23:562
9327991512,2cyclictest32-21ksoftirqd/200:00:012
9327991512,2cyclictest13312-21ls23:50:182
9327991511,3cyclictest28322-21cron19:50:002
932799150,15cyclictest0-21swapper/222:27:322
932399155,7cyclictest16717-21apt-get19:25:131
932399155,4cyclictest30729-21apt-get00:20:131
932399151,1cyclictest1860-21diskmemload23:12:461
931899159,2cyclictest7051-21wc22:30:000
931899158,2cyclictest1861-21tr23:05:140
9332991413,1cyclictest0-21swapper/323:42:043
933299141,0cyclictest0-21swapper/323:51:153
933299140,13cyclictest0-21swapper/321:13:243
933299140,13cyclictest0-21swapper/319:25:203
933299140,12cyclictest0-21swapper/300:00:133
932799149,4cyclictest32-21ksoftirqd/220:18:582
932799144,6cyclictest23126-21ssh00:35:002
932799141,12cyclictest10804-21apt-get21:15:132
932799140,1cyclictest0-21swapper/200:14:082
932799140,14cyclictest0-21swapper/222:19:502
9323991411,2cyclictest32678-21cut22:45:141
932399141,12cyclictest22547-21rm21:40:381
9323991410,3cyclictest32097-21cron22:45:001
9323991410,2cyclictest31745-21grep22:25:141
931899149,1cyclictest18388-21apt-get23:15:120
9318991413,1cyclictest0-21swapper/021:21:380
9318991410,3cyclictest6794-21grep20:10:190
9318991410,3cyclictest15388-21sed21:55:270
933299139,3cyclictest3607-21tr21:30:133
933299131,7cyclictest23148-21munin-run00:35:003
9332991311,1cyclictest39-21ksoftirqd/321:41:353
9332991310,2cyclictest31399-21sort00:20:153
9332991310,2cyclictest10708-21grep22:50:243
9332991310,1cyclictest1860-21diskmemload00:10:443
933299130,12cyclictest21709-21sed20:40:243
933299130,12cyclictest10879-21wc19:10:233
9327991311,1cyclictest32-21ksoftirqd/200:08:112
932799131,0cyclictest141rcu_preempt00:29:472
9327991310,2cyclictest31499-21cut00:20:162
9327991310,2cyclictest26422-21grep20:50:262
9327991310,2cyclictest21246-21ls20:40:202
9327991310,2cyclictest20626-21cat19:30:272
9327991310,2cyclictest19760-21cron20:40:012
9327991310,2cyclictest15322-21cat19:20:202
9327991310,2cyclictest14044-21cron19:20:002
932399139,3cyclictest19200-21sed23:15:161
932399139,3cyclictest18443-21tail20:35:191
932399131,12cyclictest1860-21diskmemload21:19:021
9323991310,2cyclictest3823-21tr21:30:151
932399130,11cyclictest23178-21cut20:45:161
931899139,3cyclictest11189-21ssh23:30:120
931899136,6cyclictest537-21snmpd21:52:100
931899130,13cyclictest0-21swapper/023:03:490
933299129,2cyclictest8356-21cut20:15:143
933299129,2cyclictest39-21ksoftirqd/322:55:223
933299129,2cyclictest29505-21sed21:25:263
933299129,2cyclictest24497-21tr22:40:123
933299129,2cyclictest23862-21ssh22:20:163
933299129,2cyclictest2366-21grep23:05:163
933299129,2cyclictest23201-21head20:45:173
933299129,2cyclictest1708-21head20:00:183
933299128,3cyclictest4001-21grep20:05:183
9332991210,1cyclictest26900-21cron20:55:003
932799129,3cyclictest0-21swapper/223:35:072
932799129,2cyclictest31519-21awk00:39:592
932799129,2cyclictest27409-21grep20:55:152
932799129,2cyclictest26346-21grep23:00:182
932799129,2cyclictest25455-21grep19:40:252
932799129,2cyclictest23456-21cat20:45:192
932799129,2cyclictest0-21swapper/221:44:322
932799128,3cyclictest22515-21cat19:35:192
9327991210,1cyclictest3847-21cat23:25:192
932799120,11cyclictest17483-21tail19:25:172
932399129,2cyclictest6647-21grep00:05:221
932399129,2cyclictest15123-21head19:20:181
932399129,2cyclictest14853-21cut00:10:161
932399129,2cyclictest14301-21grep21:55:161
932399129,2cyclictest1236-21tr20:00:171
932399129,2cyclictest11170-21sed19:10:261
932399128,2cyclictest32295-21grep21:05:151
932399128,2cyclictest32295-21grep21:05:151
932399127,3cyclictest15550-21tr20:30:121
932399120,12cyclictest0-21swapper/122:50:551
931899129,2cyclictest472-21tr22:45:140
931899129,2cyclictest4052-21tail20:05:180
931899129,2cyclictest30950-21cron22:25:000
931899129,2cyclictest28826-21sed20:55:270
931899129,2cyclictest25031-21tr20:50:150
931899129,2cyclictest19225-21cut19:30:140
931899129,2cyclictest17762-21sed22:35:270
931899129,2cyclictest17762-21sed22:35:260
931899129,2cyclictest14173-21wc20:25:220
931899129,2cyclictest1335-21ls21:05:260
931899129,2cyclictest1335-21ls21:05:260
931899128,3cyclictest24329-21grep00:15:250
931899128,3cyclictest24268-21grep22:40:120
931899128,3cyclictest20818-21cat23:35:200
931899128,3cyclictest18106-21sed22:55:170
931899128,1cyclictest85362sleep023:28:240
931899127,3cyclictest2223-21cron23:25:000
931899120,7cyclictest14835-21latency_hist00:30:000
933299119,1cyclictest32229-21tr21:05:153
933299119,1cyclictest32229-21tr21:05:143
933299119,1cyclictest31667-21sh22:05:213
933299118,2cyclictest8244-21grep22:30:153
933299118,2cyclictest6825-21ssh00:25:113
933299118,2cyclictest6117-21ssh22:10:043
933299118,2cyclictest32479-21ssh22:45:083
933299118,2cyclictest28016-21taskset19:46:053
933299118,2cyclictest26668-21cat23:00:193
933299118,2cyclictest26626-21sh00:17:133
933299118,2cyclictest25710-21ssh00:35:293
933299118,2cyclictest21834-21expr19:35:143
933299118,2cyclictest21691-21chrt23:16:283
933299118,2cyclictest20070-21wc19:30:203
933299118,2cyclictest16597-21rm23:14:003
933299118,2cyclictest14045-21cron19:20:003
933299111,3cyclictest477-21rs:main3
933299110,2cyclictest0-21swapper/323:20:263
933299110,10cyclictest28983-21chrt20:56:193
932799118,2cyclictest8588-21ssh21:51:482
932799118,2cyclictest7320-21ssh22:30:092
932799118,2cyclictest5864-21sed23:45:252
932799118,2cyclictest31685-21tail19:55:162
932799118,2cyclictest30993-21rm23:41:132
932799118,2cyclictest30959-21smart_sda21:00:232
932799118,2cyclictest3067-21ls20:04:592
932799118,2cyclictest29655-21wc19:50:182
932799118,2cyclictest26612-21ssh22:40:432
932799118,2cyclictest22701-21cat00:15:142
932799118,2cyclictest19446-21wc23:15:182
932799118,2cyclictest18409-21missed_timers22:55:192
932799118,2cyclictest17179-21ssh21:56:442
932799118,2cyclictest12623-21ssh22:51:492
932799118,2cyclictest11977-21wc20:20:232
932799118,2cyclictest10322-21grep19:10:172
932799116,4cyclictest28678-21memory21:25:182
932799111,1cyclictest141rcu_preempt22:05:032
932799110,10cyclictest17716-21cat22:35:272
932799110,10cyclictest17716-21cat22:35:262
932399118,2cyclictest8688-21ntp_states22:30:201
932399118,2cyclictest5007-21grep20:05:271
932399118,2cyclictest4877-21ssh21:10:501
932399118,2cyclictest32481-21ssh21:46:401
932399118,2cyclictest30343-21tail21:00:151
932399118,2cyclictest27870-21cut20:55:171
932399118,2cyclictest27574-21grep23:20:171
932399118,2cyclictest25710-21ssh00:35:291
932399118,2cyclictest2069-21chrt22:07:061
932399118,2cyclictest20456-21grep21:20:211
932399118,2cyclictest20362-21ssh23:35:171
932399118,2cyclictest20159-21tr20:40:131
932399118,2cyclictest14026-21taskset19:19:511
932399118,2cyclictest12365-21basename23:50:121
932399118,2cyclictest10108-21taskset20:18:171
932399110,2cyclictest21761-21apt-config22:00:001
932399110,1cyclictest1860-21diskmemload23:29:021
932399110,10cyclictest30059-21sed19:50:241
932399110,10cyclictest28233-21taskset19:48:531
931899118,2cyclictest8115-21ssh00:06:110
931899118,2cyclictest6103-21ssh21:11:480
931899118,2cyclictest31536-21wc22:05:210
931899118,2cyclictest30150-21cat00:00:180
931899118,2cyclictest24571-21chrt22:00:470
931899118,2cyclictest21158-21head20:40:180
931899118,2cyclictest19355-21head20:35:280
931899118,2cyclictest18420-21ssh23:53:340
931899118,2cyclictest18242-21rm00:12:110
931899118,2cyclictest16782-21cut19:25:140
931899118,2cyclictest16004-21tail20:30:140
931899118,2cyclictest1549-21sort20:00:190
931899118,2cyclictest14475-21cut19:20:140
931899118,2cyclictest14324-21ssh22:53:090
931899118,2cyclictest14286-21sh22:34:100
931899117,3cyclictest13071-21tr19:15:200
931899116,4cyclictest29211-21sed23:40:190
931899114,5cyclictest0-21swapper/022:10:180
931899110,10cyclictest4166-21expr23:45:130
931899110,10cyclictest3268-21ssh21:30:030
933299108,1cyclictest31237-21cut19:55:143
933299108,1cyclictest17109-21taskset20:31:273
933299108,1cyclictest14691-21expr19:20:153
933299108,1cyclictest13060-21egrep20:25:133
933299107,2cyclictest7592-21seq20:12:253
933299107,2cyclictest30289-21grep21:00:153
933299107,2cyclictest29039-21expr19:50:143
933299107,2cyclictest24062-21cat19:40:143
933299107,2cyclictest17698-21wget20:35:163
933299107,2cyclictest11410-21cat20:20:173
932799108,1cyclictest17182-21taskset20:32:252
932799108,1cyclictest13194-21grep20:25:152
932799108,1cyclictest10006-21ssh23:10:132
932799107,2cyclictest7954-21wc20:15:012
932799107,2cyclictest3322-21apache_accesses20:05:122
932799107,2cyclictest31963-21gpgv21:05:012
932799107,2cyclictest31963-21gpgv21:05:002
932399107,2cyclictest7229-21ssh21:50:441
932399107,2cyclictest13668-21sed20:25:171
931899108,1cyclictest30775-21cut21:00:200
931899107,2cyclictest9518-21taskset19:10:030
931899107,2cyclictest27079-21mii-tool19:45:180
93239996,2cyclictest24041-21grep19:40:141
93239996,2cyclictest19189-21wget19:30:141
93239994,2cyclictest452-21systemd-logind20:50:241
93189997,1cyclictest8694-21grep20:15:150
93189997,1cyclictest31532-21sed19:55:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional