You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 23:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu May 28, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1193799370,32cyclictest151rcu_preempt10:08:152
11937992725,1cyclictest28-21ksoftirqd/209:05:012
1193799262,0cyclictest28-21ksoftirqd/211:05:232
11938992523,2cyclictest11615-21ssh12:15:523
11938992523,1cyclictest23963-21apt-get12:28:153
1193899250,1cyclictest479-21rm12:05:473
11937992523,1cyclictest0-21swapper/210:11:042
11937992522,1cyclictest28-21ksoftirqd/212:40:002
11937992522,0cyclictest28-21ksoftirqd/211:57:432
11937992520,4cyclictest0-21swapper/212:05:362
1193799250,24cyclictest0-21swapper/212:01:332
1193699250,25cyclictest0-21swapper/112:14:121
1193699250,24cyclictest6538-21diskmemload09:53:591
1193599253,0cyclictest62550irq/127-eno109:28:270
1193599250,24cyclictest0-21swapper/009:47:180
11938992422,1cyclictest1075-21sh10:35:313
1193899240,24cyclictest0-21swapper/310:01:353
1193899240,23cyclictest6538-21diskmemload12:39:533
1193899240,23cyclictest28313-21rm12:31:413
1193899240,23cyclictest0-21swapper/310:30:533
1193899240,1cyclictest6538-21diskmemload12:02:523
1193899240,1cyclictest6538-21diskmemload09:37:503
11937992423,1cyclictest6538-21diskmemload11:37:152
11937992422,1cyclictest0-21swapper/211:26:242
11937992422,1cyclictest0-21swapper/211:21:052
11937992422,1cyclictest0-21swapper/211:15:212
11937992422,0cyclictest0-21swapper/210:02:512
11937992420,1cyclictest17993-21kworker/u8:109:39:252
1193799240,23cyclictest6816-21ssh11:41:442
1193799240,23cyclictest29073-21ssh10:31:502
1193799240,23cyclictest0-21swapper/210:52:152
11936992424,0cyclictest0-21swapper/109:20:001
11936992421,2cyclictest29239-21sh12:33:061
1193699240,24cyclictest0-21swapper/110:50:371
1193699240,23cyclictest6538-21diskmemload09:34:001
11935992423,1cyclictest62550irq/127-eno110:30:380
11935992423,1cyclictest16807-21ssh09:19:160
11935992421,2cyclictest0-21swapper/010:07:330
1193599240,24cyclictest0-21swapper/012:12:190
1193599240,24cyclictest0-21swapper/010:56:140
1193599240,23cyclictest6538-21diskmemload11:11:400
1193599240,23cyclictest0-21swapper/011:53:150
1193599240,0cyclictest6538-21diskmemload10:45:020
1193599240,0cyclictest0-21swapper/009:20:580
11938992323,0cyclictest6538-21diskmemload11:23:213
11938992323,0cyclictest6538-21diskmemload10:25:133
11938992323,0cyclictest24693-21ssh10:58:153
11938992323,0cyclictest0-21swapper/311:08:303
11938992323,0cyclictest0-21swapper/310:21:493
11938992323,0cyclictest0-21swapper/310:09:183
11938992323,0cyclictest0-21swapper/309:55:253
11938992321,1cyclictest0-21swapper/311:55:013
11938992320,2cyclictest7584-21ssh09:41:123
11938992320,2cyclictest23561-21ssh09:25:243
1193899231,1cyclictest10719-21turbostat11:45:003
1193899230,23cyclictest0-21swapper/311:48:593
1193899230,23cyclictest0-21swapper/311:37:403
1193899230,23cyclictest0-21swapper/309:50:203
1193899230,22cyclictest6538-21diskmemload10:15:113
1193899230,22cyclictest28707-21ssh09:30:143
1193899230,22cyclictest0-21swapper/311:02:073
1193899230,22cyclictest0-21swapper/310:11:533
1193899230,0cyclictest0-21swapper/312:24:443
1193899230,0cyclictest0-21swapper/311:33:163
1193899230,0cyclictest0-21swapper/311:17:453
1193899230,0cyclictest0-21swapper/309:12:523
11937992323,0cyclictest0-21swapper/211:54:142
11937992323,0cyclictest0-21swapper/211:11:412
11937992323,0cyclictest0-21swapper/210:55:162
11937992323,0cyclictest0-21swapper/209:51:202
11937992323,0cyclictest0-21swapper/207:33:142
11937992321,1cyclictest0-21swapper/212:22:402
11937992321,1cyclictest0-21swapper/212:15:022
11937992321,1cyclictest0-21swapper/210:27:282
1193799232,0cyclictest28-21ksoftirqd/209:49:482
11937992318,1cyclictest11416-21ssh09:14:182
1193799231,1cyclictest0-21swapper/209:27:102
1193799230,2cyclictest23775-21ssh09:56:242
1193799230,2cyclictest17606-21sh09:20:032
1193799230,23cyclictest0-21swapper/210:16:502
1193799230,23cyclictest0-21swapper/209:34:092
1193799230,22cyclictest7622-21rm09:41:152
1193799230,22cyclictest4514-21tr08:00:002
1193799230,22cyclictest30028-21gltestperf12:33:192
1193799230,0cyclictest0-21swapper/210:40:372
1193799230,0cyclictest0-21swapper/209:16:072
11936992323,0cyclictest6538-21diskmemload12:07:041
11936992323,0cyclictest6369-21ssh11:10:201
11936992323,0cyclictest0-21swapper/111:27:041
11936992323,0cyclictest0-21swapper/111:21:111
11936992323,0cyclictest0-21swapper/111:16:341
11936992323,0cyclictest0-21swapper/111:06:261
11936992323,0cyclictest0-21swapper/110:47:581
11936992323,0cyclictest0-21swapper/109:48:241
11936992323,0cyclictest0-21swapper/109:42:041
11936992323,0cyclictest0-21swapper/109:14:591
11936992323,0cyclictest0-21swapper/107:35:021
11936992322,1cyclictest0-21swapper/112:04:161
11936992322,1cyclictest0-21swapper/109:55:191
11936992321,1cyclictest23406-21sh10:56:261
11936992321,1cyclictest0-21swapper/111:45:191
1193699230,22cyclictest3802-21memory10:38:201
1193699230,22cyclictest28395-21ssh11:31:281
1193699230,22cyclictest0-21swapper/112:38:511
1193699230,22cyclictest0-21swapper/111:51:561
1193699230,1cyclictest1504-21ssh10:05:311
1193699230,1cyclictest12375-21cron12:17:001
1193699230,0cyclictest0-21swapper/112:27:011
1193699230,0cyclictest0-21swapper/111:59:451
1193699230,0cyclictest0-21swapper/109:39:101
1193699230,0cyclictest0-21swapper/109:21:361
11935992323,0cyclictest0-21swapper/010:10:230
11935992322,1cyclictest6538-21diskmemload11:30:190
11935992322,1cyclictest6538-21diskmemload09:13:540
11935992322,1cyclictest22696-21ssh10:25:160
11935992321,1cyclictest1407-21ssh10:36:010
11935992321,1cyclictest0-21swapper/012:22:130
11935992320,2cyclictest349-21sh12:05:360
11935992320,2cyclictest12356-21ssh10:15:470
1193599230,2cyclictest22095-21ssh12:25:270
1193599230,2cyclictest1772-21sendmail-msp09:00:010
1193599230,2cyclictest1772-21sendmail-msp09:00:010
1193599230,23cyclictest0-21swapper/009:55:330
1193599230,22cyclictest62550irq/127-eno109:35:290
1193599230,22cyclictest32380-21sh11:04:120
1193599230,22cyclictest19497-21sh10:23:150
1193599230,22cyclictest16848-21diskmemload11:20:020
1193599230,21cyclictest0-21swapper/011:05:370
1193599230,0cyclictest6538-21diskmemload10:40:520
1193599230,0cyclictest0-21swapper/012:39:020
1193599230,0cyclictest0-21swapper/011:28:220
1193599230,0cyclictest0-21swapper/010:50:030
1193599230,0cyclictest0-21swapper/009:51:470
11938992222,0cyclictest6538-21diskmemload11:25:513
11938992222,0cyclictest0-21swapper/312:11:373
11938992222,0cyclictest0-21swapper/310:51:143
11938992220,1cyclictest12399-21ssh09:45:183
1193899220,21cyclictest29880-21apt-get08:53:163
1193899220,21cyclictest0-21swapper/311:10:243
1193899220,21cyclictest0-21swapper/309:08:153
1193899220,1cyclictest6538-21diskmemload11:52:233
1193899220,0cyclictest6538-21diskmemload10:40:083
1193899220,0cyclictest0-21swapper/310:45:343
11937992222,0cyclictest0-21swapper/212:10:032
11937992222,0cyclictest0-21swapper/211:30:272
11937992220,1cyclictest0-21swapper/210:45:022
11937992220,1cyclictest0-21swapper/210:35:022
1193799220,21cyclictest6538-21diskmemload12:25:412
1193799220,21cyclictest6538-21diskmemload11:45:032
1193799220,21cyclictest16795-21idleruntime-cro07:20:012
1193799220,21cyclictest15090-21apt-get08:23:162
1193799220,1cyclictest28144-21ssh11:00:262
1193799220,0cyclictest151rcu_preempt10:22:042
11936992222,0cyclictest0-21swapper/110:26:441
11936992222,0cyclictest0-21swapper/110:22:571
11936992220,1cyclictest6922-21awk08:05:001
11936992220,1cyclictest0-21swapper/111:37:561
1193699220,22cyclictest0-21swapper/111:02:331
1193699220,21cyclictest6538-21diskmemload11:40:061
1193699220,21cyclictest6538-21diskmemload10:15:261
1193699220,21cyclictest4671-21apt-get09:08:151
1193699220,21cyclictest0-21swapper/112:22:151
1193699220,21cyclictest0-21swapper/110:00:021
1193699220,21cyclictest0-21swapper/108:15:011
1193699220,21cyclictest0-21swapper/108:05:021
1193699220,1cyclictest6538-21diskmemload10:12:001
1193699220,1cyclictest6428-21ssh10:40:341
1193699220,1cyclictest21588-21grep07:30:011
1193699220,0cyclictest6538-21diskmemload09:26:151
1193699220,0cyclictest0-21swapper/110:31:231
11935992222,0cyclictest6538-21diskmemload12:03:350
11935992222,0cyclictest6538-21diskmemload11:35:200
11935992221,0cyclictest0-21swapper/011:42:150
11935992220,1cyclictest6538-21diskmemload12:30:280
11935992220,1cyclictest12016-21sh11:15:530
1193599220,2cyclictest28793-21ssh10:00:490
1193599220,22cyclictest0-21swapper/011:56:350
1193599220,22cyclictest0-21swapper/011:45:300
1193599220,21cyclictest2279-21apt-get09:03:150
1193599220,21cyclictest0-21swapper/008:03:160
1193599220,1cyclictest9696-21awk08:10:010
1193599220,1cyclictest27481-21apt-get08:48:160
11938992119,1cyclictest10214-21apt-get08:13:153
1193899210,1cyclictest29363-21awk08:50:003
11937992120,0cyclictest26860-21apt-get07:40:012
1193799210,20cyclictest0-21swapper/209:08:152
1193799210,1cyclictest17483-21apt-get08:28:162
11936992120,0cyclictest0-21swapper/107:18:171
1193699210,20cyclictest0-21swapper/107:58:171
1193699210,0cyclictest0-21swapper/108:18:171
11935992120,1cyclictest32583-21apt-get07:53:160
11935992120,1cyclictest14883-21apt-get07:18:170
1193599210,21cyclictest0-21swapper/007:48:160
1193599210,0cyclictest0-21swapper/007:40:010
11938992020,0cyclictest12675-21apt-get08:18:173
11938992020,0cyclictest0-21swapper/308:08:153
11938992020,0cyclictest0-21swapper/307:23:153
11937992020,0cyclictest0-21swapper/207:48:162
11937992019,1cyclictest0-21swapper/208:48:162
11937992018,1cyclictest6886-21cron08:05:002
11937992018,1cyclictest606-21dbus-daemon07:40:002
11936992020,0cyclictest0-21swapper/108:38:161
11936992019,0cyclictest0-21swapper/108:40:021
11936992019,0cyclictest0-21swapper/107:48:161
1193699200,0cyclictest0-21swapper/107:13:161
11935992020,0cyclictest0-21swapper/008:28:160
11935992020,0cyclictest0-21swapper/008:18:170
11935992020,0cyclictest0-21swapper/008:13:150
11935992017,2cyclictest14362-21apt07:15:000
11938991919,0cyclictest0-21swapper/308:28:163
11938991919,0cyclictest0-21swapper/307:58:173
11938991919,0cyclictest0-21swapper/307:18:173
11937991918,0cyclictest0-21swapper/208:08:152
11937991917,1cyclictest24187-21grep08:40:002
11936991919,0cyclictest0-21swapper/109:03:151
11936991919,0cyclictest0-21swapper/107:23:151
11935991919,0cyclictest19882-21apt-get08:33:170
11935991918,0cyclictest0-21swapper/007:58:170
11938991818,0cyclictest0-21swapper/307:53:163
11938991818,0cyclictest0-21swapper/307:35:023
11938991816,1cyclictest0-21swapper/307:30:013
11937991818,0cyclictest0-21swapper/208:33:172
11937991818,0cyclictest0-21swapper/208:13:162
11937991817,0cyclictest0-21swapper/208:43:182
11936991818,0cyclictest0-21swapper/108:53:161
11936991817,0cyclictest0-21swapper/108:48:161
11936991816,1cyclictest27791-21apt-get07:43:161
11935991818,0cyclictest0-21swapper/009:08:150
11935991818,0cyclictest0-21swapper/008:23:160
11935991818,0cyclictest0-21swapper/007:28:160
11938991716,1cyclictest2245-21sshd09:03:143
11935991717,0cyclictest0-21swapper/008:40:020
11935991716,0cyclictest17296-21apt-get07:23:150
11938991616,0cyclictest30186-21apt-get07:48:163
11938991614,1cyclictest22609-21perl07:33:163
11938991614,1cyclictest12486-21apt-get07:13:163
11937991614,1cyclictest14592-21sendmail-msp08:20:012
1193699160,1cyclictest19347-21cron08:30:001
11935991614,1cyclictest0-21swapper/008:38:190
1193899150,15cyclictest6985-21dpkg08:05:013
11937991515,0cyclictest0-21swapper/207:28:162
11936991515,0cyclictest22086-21apt-get07:33:141
11938991414,0cyclictest0-21swapper/308:38:103
11938991414,0cyclictest0-21swapper/308:30:123
1193899140,14cyclictest0-21swapper/308:55:083
1193899140,14cyclictest0-21swapper/308:55:083
1193899140,14cyclictest0-21swapper/308:41:033
1193899140,14cyclictest0-21swapper/308:20:423
1193899140,14cyclictest0-21swapper/307:40:553
11937991411,2cyclictest1-21systemd07:25:012
1193799140,14cyclictest0-21swapper/208:55:562
1193799140,14cyclictest0-21swapper/208:55:562
1193799140,14cyclictest0-21swapper/208:51:262
11936991414,0cyclictest0-21swapper/108:31:001
11936991414,0cyclictest0-21swapper/108:20:161
11936991414,0cyclictest0-21swapper/107:50:061
1193699140,0cyclictest0-21swapper/108:59:201
1193699140,0cyclictest0-21swapper/108:59:201
11935991414,0cyclictest0-21swapper/008:50:210
11935991414,0cyclictest0-21swapper/007:35:110
1193599140,14cyclictest0-21swapper/007:30:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional